turicas/rows

Detect data type from custom fields

Open

#271 opened on Mar 20, 2018

 (0 comments) (0 reactions) (0 assignees)Python (136 forks)github user discovery
enhancementhelp wantedpluginrows.fieldsrows.table

Repository metrics

Stars
 (886 stars)
PR merge metrics
 (No merged PRs in 30d)

Description

I have this schema, with some custom fields:

import rows


class MoneyField(rows.fields.TextField):

    @classmethod
    def deserialize(cls, value):
        value = value.replace('.', '')\
                     .replace(',', '.')
        return super().deserialize(value)


class DocumentField(rows.fields.TextField):

    @classmethod
    def deserialize(cls, value):
        value = value.replace(' ', '')\
                     .replace('.', '')\
                     .replace('-', '').strip()
        return super().deserialize(value)

schema = OrderedDict([
        ('arquivoOrigem', rows.fields.TextField),
        ('codLegislatura', rows.fields.IntegerField),
        ('datEmissao', rows.fields.DatetimeField),
        ('ideDocumento', rows.fields.IntegerField),
        ('idecadastro', rows.fields.IntegerField),
        ('indTipoDocumento', rows.fields.IntegerField),
        ('nuCarteiraParlamentar', rows.fields.IntegerField),
        ('nuDeputadoId', rows.fields.IntegerField),
        ('nuLegislatura', rows.fields.IntegerField),
        ('numAno', rows.fields.IntegerField),
        ('numEspecificacaoSubCota', rows.fields.IntegerField),
        ('numLote', rows.fields.IntegerField),
        ('numMes', rows.fields.IntegerField),
        ('numParcela', rows.fields.IntegerField),
        ('numRessarcimento', rows.fields.IntegerField),
        ('numSubCota', rows.fields.IntegerField),
        ('sgPartido', rows.fields.TextField),
        ('sgUF', rows.fields.TextField),
        ('txNomeParlamentar', rows.fields.TextField),
        ('txtCNPJCPF', DocumentField),
        ('txtDescricao', rows.fields.TextField),
        ('txtDescricaoEspecificacao', rows.fields.TextField),
        ('txtFornecedor', rows.fields.TextField),
        ('txtNumero', rows.fields.TextField),
        ('txtPassageiro', rows.fields.TextField),
        ('txtTrecho', rows.fields.TextField),
        ('vlrDocumento', MoneyField),
        ('vlrGlosa', MoneyField),
        ('vlrLiquido', MoneyField),
        ('vlrRestituicao', MoneyField),
])

It works when I import data using it (rows.Table(fields=schema)) but won't work in an expected way if I try to export this table, because the library does not know how to export MoneyField and DocumentField to SQLite (only knows rows.fields.*Field classes). As a user, to fix this I needed to have a specific schema for the Table object, like this:

    def convert_field(FieldClass):
        if FieldClass is MoneyField:
            return rows.fields.DecimalField
        elif FieldClass is DocumentField:
            return rows.fields.TextField
        else:
            return FieldClass
    schema_rows = OrderedDict([(field_name, convert_field(Field))
                               for field_name, Field in schema.items()])

The library should detect from the values produced by the class or by inspecting it.

Contributor guide