antu.io.fields package¶
Submodules¶
antu.io.fields.field module¶
-
class
antu.io.fields.field.Field[source]¶ Bases:
objectA
Fieldis an ingredient of a data instance. In most NLP tasks,Fieldstores data of string types. It contains one or more indexers that map string data to the corresponding index. Data instances are collections of fields.Methods
count_vocab_items(counter, Dict[str, int]])We count the number of strings if the string needs to be mapped to one or more integers. index(vocab)Gets one or more index mappings for each element in the Field. -
count_vocab_items(counter: Dict[str, Dict[str, int]]) → None[source]¶ We count the number of strings if the string needs to be mapped to one or more integers. You can pass directly if there is no string that needs to be mapped.
Parameters: - counter :
Dict[str, Dict[str, int]] - ``counter`` is used to count the number of each item. The first key
- represents the namespace of the vocabulary, and the second key represents
- the string of the item.
- counter :
-
antu.io.fields.index_field module¶
-
class
antu.io.fields.index_field.IndexField(name: str, tokens: List[str])[source]¶ Bases:
antu.io.fields.field.FieldA
IndexFieldis an integer field, and we can use it to store data ID.Parameters: - name :
str Field name. This is necessary and must be unique (not the same as other field names).
- tokens :
List[str] Field content that contains a list of string.
Methods
count_vocab_items(counters, Dict[str, int]])IndexFielddoesn’t need index operation.index(vocab)IndexFielddoesn’t need index operation.- name :
antu.io.fields.sequence_label_field module¶
-
class
antu.io.fields.sequence_label_field.SequenceLabelField(name: str, tokens: List[str], indexers: List[antu.io.token_indexers.token_indexer.TokenIndexer])[source]¶ Bases:
antu.io.fields.field.FieldMethods
count_vocab_items(counters, Dict[str, int]])We count the number of strings if the string needs to be mapped to one or more integers. index(vocab)Gets one or more index mappings for each element in the Field. -
count_vocab_items(counters: Dict[str, Dict[str, int]]) → None[source]¶ We count the number of strings if the string needs to be mapped to one or more integers. You can pass directly if there is no string that needs to be mapped.
Parameters: - counter :
Dict[str, Dict[str, int]] - ``counter`` is used to count the number of each item. The first key
- represents the namespace of the vocabulary, and the second key represents
- the string of the item.
- counter :
-
antu.io.fields.text_field module¶
-
class
antu.io.fields.text_field.TextField(name: str, tokens: List[str], indexers: List[antu.io.token_indexers.token_indexer.TokenIndexer] = [])[source]¶ Bases:
antu.io.fields.field.FieldA
TextFieldis a data field that is commonly used in NLP tasks, and we can use it to store text sequences such as sentences, paragraphs, POS tags, and so on.Parameters: - name :
str Field name. This is necessary and must be unique (not the same as other field names).
- tokens :
List[str] Field content that contains a list of string.
- indexers :
List[TokenIndexer], optional (default=``list()``) Indexer list that defines the vocabularies associated with the field.
Methods
count_vocab_items(counters, Dict[str, int]])We count the number of strings if the string needs to be counted to some index(vocab)Gets one or more index mappings for each element in the Field. -
count_vocab_items(counters: Dict[str, Dict[str, int]]) → None[source]¶ - We count the number of strings if the string needs to be counted to some
- counters. You can pass directly if there is no string that needs
to be counted.
Parameters: - counters :
Dict[str, Dict[str, int]] Element statistics for datasets. if field indexers indicate that this field is related to some counters, we use field content to update the counters.
- name :