String Functions
Functions for string manipulation, text processing, and string analysis.
Usage
String functions are available under the func.string namespace to avoid name collisions with other functions:
string
byte_hamming_distance
Computes the Hamming distance between two strings.
The Hamming distance is the number of positions at which the corresponding characters are different. This function returns the dissimilarity between the strings, where 0 indicates identical strings and values closer to the length of the strings indicate higher dissimilarity.
Parameters:
-
args(str | Column | Func | literal, default:()) –Two strings to compute the Hamming distance between. If a string is provided, it is assumed to be the name of the column. If a Column is provided, it is assumed to be a column in the dataset. If a Func is provided, it is assumed to be a function returning a string. If a literal is provided, it is assumed to be a string literal.
Returns:
-
Func(Func) –A
Funcobject that represents the Hamming distance function.
Example
Notes
- The result column will always be of type int.
Source code in datachain/func/string.py
length
Returns the length of the string.
Parameters:
-
col(str | Column | Func | literal) –String to compute the length of. If a string is provided, it is assumed to be the name of the column. If a Column is provided, it is assumed to be a column in the dataset. If a Func is provided, it is assumed to be a function returning a string. If a literal is provided, it is assumed to be a string literal.
Returns:
-
Func(Func) –A
Funcobject that represents the string length function.
Example
Notes
- The result column will always be of type int.
Source code in datachain/func/string.py
regexp_replace
Replaces substring that match a regular expression.
Parameters:
-
col(str | Column | Func | literal) –Column to perform replacement on. If a string is provided, it is assumed to be the name of the column. If a Column is provided, it is assumed to be a column in the dataset. If a Func is provided, it is assumed to be a function returning a string. If a literal is provided, it is assumed to be a string literal.
-
regex(str) –Regular expression pattern to replace.
-
replacement(str) –Replacement string.
Returns:
-
Func(Func) –A
Funcobject that represents the regexp_replace function.
Example
Notes
- The result column will always be of type string.
- Matching is case-sensitive by default. For case-insensitive matching
prepend the
(?i)inline flag to the pattern (e.g.r"(?i)foo").
Source code in datachain/func/string.py
replace
Replaces substring with another string.
Parameters:
-
col(str | Column | Func | literal) –Column to perform replacement on. If a string is provided, it is assumed to be the name of the column. If a Column is provided, it is assumed to be a column in the dataset. If a Func is provided, it is assumed to be a function returning a string. If a literal is provided, it is assumed to be a string literal.
-
pattern(str) –Pattern to replace.
-
replacement(str) –Replacement string.
Returns:
-
Func(Func) –A
Funcobject that represents the replace function.
Example
Notes
- The result column will always be of type string.
Source code in datachain/func/string.py
split
Takes a column and split character and returns an array of the parts.
Parameters:
-
col(str | Column | Func | literal) –Column to split. If a string is provided, it is assumed to be the name of the column. If a Column is provided, it is assumed to be a column in the dataset. If a Func is provided, it is assumed to be a function returning a string. If a literal is provided, it is assumed to be a string literal.
-
sep(str) –Separator to split the string.
-
limit(int, default:None) –Maximum number of splits to perform.
Returns:
-
Func(Func) –A
Funcobject that represents the split function.
Example
Notes
- The result column will always be of type array of strings.
Source code in datachain/func/string.py
string_hash
Computes a 64-bit hash of one or more column values.
Values are coerced to strings and concatenated with '|' separator
before hashing. Useful as input to func.xor_agg for computing
order-independent content fingerprints.
Parameters:
-
*cols(ColT, default:()) –One or more columns to hash.
Returns:
-
Func(Func) –A
Funcobject that represents the string_hash function.