simonw/sqlite-utils

Misleading progress bar against utf-16-le CSV input

Open

#439 opened on May 27, 2022

View on GitHubContributor guide
 (12 comments) (0 reactions) (0 assignees)Python (145 forks)github user discovery
bughelp wanted

Repository metrics

Stars
 (2,065 stars)
PR merge metrics
 (Avg merge 9h 52m) (7 merged PRs in 30d)

Description

The program crashes without any error.

wget "https://artsdatabanken.no/Fab2018/api/export/csv"
sqlite-utils create-database test.db
sqlite-utils insert --csv --delimiter ";" --encoding "utf-16-le" test test.db csv 
  [------------------------------------]    0%
  [#################-------------------]   49%  00:00:01

I would like to highlight various issues:

  1. sqlite-utils catches exceptions without printing the stacktrace and/or reraising the exception, so there is no easy way to use pdb or similar to debug the program, solution: add a debug option
  2. Silent crash: this is related to (1.), and it happens when there is a catch-all mechanism; solution: let the program fail.

Contributor guide

Research direction
Reproduce the issue with an utf 16 le CSV file and trace the stack trace to identify where the silent crash occurs. Then implement a debug flag to show errors and optionally let the program fail with a proper error message.
Tech stack
python
Domain
clidatabase
Issue type
Bug
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
2
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-3 hours
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
PythonGit
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
70