Find here DataChain news, findings, interesting reads, community takeaways, deep dive into machine learning workflows from data versioning and processing to model productionization.
OpenAI built their in-house data agent for structured warehouse data, where schema, lineage, and queries come for free. Files in S3, GCS, or Azure - videos, sensor logs, image corpora, PDFs - have none of that, and the problems get a lot more interesting. Here is how we built the four foundations that close the gap.
Dmitry Petrov
Extract and parse text from documents and create vector embeddings in a scalable and distributed way (and less than 70 lines of code).
Tibor Mach
Learn how to fine tune multimodal models like CLIP to match images to text captions.
Dave Berenbaum
The results of our tests on the structured outputs of Google Gemini Pro, Anthropic Claude, and OpenAI GPT. DataChain used for evaluation.
Daniel Kharitonov
Introducing DataChain - a new open-source tool to curate and process unstructured data using local ML models, and LLM calls.
Dmitry Petrov
Learn about our latest approach to mastering your Unstructured Data and metadata.
Jeny De Figueiredo, Daniel Kharitonov
