Datachain

VideoFile

VideoFile extends File and provides additional methods for working with video files.

VideoFile instances are created when a DataChain is initialized from storage with the type="video" parameter:

import datachain as dc

chain = dc.read_storage("s3://bucket-name/", type="video")

There are additional models for working with video files:

  • VideoFrame - represents a single frame of a video file, including its video stream index, frame index, and timestamp.
  • VideoFragment - represents a fragment of a video file.

video_stream_index arguments are zero-based indexes among video streams, matching FFmpeg v:N and PyAV container.streams.video[N] selectors.

VideoFile.get_frame() returns a single frame reference with a timestamp estimated from FPS metadata. Pixel access may seek to a nearby keyframe and decode forward; use VideoFile.get_frames() for sequential access and decoded frame timestamps when available.

These are virtual models that do not create physical files. Instead, they are used to represent the data in the VideoFile these models are referring to. If you need to save the data, you can use the save method of these models, allowing you to save data locally or upload it to a storage service.

VideoFile

VideoFile(**kwargs)

Bases: File

A data model for handling video files.

This model inherits from the File model and provides additional functionality for reading video files, extracting video frames, and splitting videos into fragments.

The video_stream_index argument used by video methods is the zero-based index among video streams, matching FFmpeg v:N and PyAV container.streams.video[N] selectors.

Source code in datachain/lib/file.py
1317
1318
1319
def __init__(self, **kwargs):
    super().__init__(**kwargs)
    self._video_info_cache: dict[int, Any] = {}

get_fragment

get_fragment(start: float, end: float) -> VideoFragment

Returns a video fragment from the specified time range.

Parameters:

  • start (float) –

    The start time of the fragment in seconds.

  • end (float) –

    The end time of the fragment in seconds.

Returns:

  • VideoFragment ( VideoFragment ) –

    A Model representing the video fragment.

Source code in datachain/lib/file.py
1403
1404
1405
1406
1407
1408
1409
1410
1411
1412
1413
1414
1415
1416
1417
1418
1419
1420
def get_fragment(self, start: float, end: float) -> "VideoFragment":
    """
    Returns a video fragment from the specified time range.

    Args:
        start (float): The start time of the fragment in seconds.
        end (float): The end time of the fragment in seconds.

    Returns:
        VideoFragment: A Model representing the video fragment.
    """
    if start < 0 or end < 0 or start >= end:
        raise ValueError(
            f"Can't get video fragment for '{self.path}', "
            f"invalid time range: ({start:.3f}, {end:.3f})"
        )

    return VideoFragment(video=self, start=start, end=end)

get_fragments

get_fragments(duration: float, start: float = 0, end: float | None = None) -> Iterator[VideoFragment]

Splits the video into multiple fragments of a specified duration.

Parameters:

  • duration (float) –

    The duration of each video fragment in seconds.

  • start (float, default: 0 ) –

    The starting time in seconds (default: 0).

  • end (float, default: None ) –

    The ending time in seconds. If None, the entire remaining video is processed (default: None).

Returns:

Note

If end is not specified, duration will be taken from the video file, which means video metadata needs to be read.

Source code in datachain/lib/file.py
1422
1423
1424
1425
1426
1427
1428
1429
1430
1431
1432
1433
1434
1435
1436
1437
1438
1439
1440
1441
1442
1443
1444
1445
1446
1447
1448
1449
1450
1451
1452
1453
1454
1455
1456
1457
1458
1459
def get_fragments(
    self,
    duration: float,
    start: float = 0,
    end: float | None = None,
) -> "Iterator[VideoFragment]":
    """
    Splits the video into multiple fragments of a specified duration.

    Args:
        duration (float): The duration of each video fragment in seconds.
        start (float): The starting time in seconds (default: 0).
        end (float, optional): The ending time in seconds. If None, the entire
                               remaining video is processed (default: None).

    Returns:
        Iterator[VideoFragment]: An iterator yielding video fragments.

    Note:
        If end is not specified, duration will be taken from the video file,
        which means video metadata needs to be read.
    """
    if duration <= 0:
        raise ValueError("duration must be a positive float")
    if start < 0:
        raise ValueError("start must be a non-negative float")

    if end is None:
        end = self.get_info().duration

    if end < 0:
        raise ValueError("end must be a non-negative float")
    if start >= end:
        raise ValueError("start must be less than end")

    while start < end:
        yield self.get_fragment(start, min(start + duration, end))
        start += duration

get_frame

get_frame(frame: int, video_stream_index: int = 0) -> VideoFrame

Returns a specific video frame by its frame number.

This returns a frame reference without decoding or validating that the frame exists. The returned timestamp is estimated from FPS metadata. Pixel access methods decode the requested frame; use get_frames() for sequential access and decoded frame timestamps when available.

Parameters:

  • frame (int) –

    The frame number to read.

  • video_stream_index (int, default: 0 ) –

    Zero-based index among video streams to read. Defaults to 0.

Returns:

  • VideoFrame ( VideoFrame ) –

    Video frame model.

Source code in datachain/lib/file.py
1343
1344
1345
1346
1347
1348
1349
1350
1351
1352
1353
1354
1355
1356
1357
1358
1359
1360
1361
1362
1363
1364
1365
def get_frame(self, frame: int, video_stream_index: int = 0) -> "VideoFrame":
    """
    Returns a specific video frame by its frame number.

    This returns a frame reference without decoding or validating that the
    frame exists. The returned timestamp is estimated from FPS metadata.
    Pixel access methods decode the requested frame; use ``get_frames()``
    for sequential access and decoded frame timestamps when available.

    Args:
        frame (int): The frame number to read.
        video_stream_index: Zero-based index among video streams to read.
            Defaults to 0.

    Returns:
        VideoFrame: Video frame model.
    """
    if frame < 0:
        raise ValueError("frame must be a non-negative integer")

    from .video import video_frame

    return video_frame(self, frame, video_stream_index=video_stream_index)

get_frames

get_frames(start: int = 0, end: int | None = None, step: int = 1, video_stream_index: int = 0) -> Iterator[VideoFrame]

Returns video frames from the specified range in the video.

Parameters:

  • start (int, default: 0 ) –

    The starting frame number (default: 0).

  • end (int, default: None ) –

    The ending frame number (exclusive). If None, frames are read until the end of the video (default: None).

  • step (int, default: 1 ) –

    The interval between frames to read (default: 1).

  • video_stream_index (int, default: 0 ) –

    Zero-based index among video streams to read. Defaults to 0.

Returns:

Note

If end is not specified, number of frames will be taken from the video file, this means video metadata needs to be read.

Source code in datachain/lib/file.py
1367
1368
1369
1370
1371
1372
1373
1374
1375
1376
1377
1378
1379
1380
1381
1382
1383
1384
1385
1386
1387
1388
1389
1390
1391
1392
1393
1394
1395
1396
1397
1398
1399
1400
1401
def get_frames(
    self,
    start: int = 0,
    end: int | None = None,
    step: int = 1,
    video_stream_index: int = 0,
) -> "Iterator[VideoFrame]":
    """
    Returns video frames from the specified range in the video.

    Args:
        start (int): The starting frame number (default: 0).
        end (int, optional): The ending frame number (exclusive). If None,
                             frames are read until the end of the video
                             (default: None).
        step (int): The interval between frames to read (default: 1).
        video_stream_index: Zero-based index among video streams to read.
            Defaults to 0.

    Returns:
        Iterator[VideoFrame]: An iterator yielding video frames.

    Note:
        If end is not specified, number of frames will be taken from the video file,
        this means video metadata needs to be read.
    """
    from .video import validate_frame_range, video_frames

    start, end, step = validate_frame_range(
        self, start, end, step, video_stream_index=video_stream_index
    )

    yield from video_frames(
        self, start, end, step, video_stream_index=video_stream_index
    )

get_info

get_info(video_stream_index: int = 0) -> Video

Retrieves metadata and information about the video file.

With prefetch and cache off, reads only the parts of the file it needs.

Parameters:

  • video_stream_index (int, default: 0 ) –

    Zero-based index among video streams to inspect. Defaults to 0.

Returns:

  • Video ( Video ) –

    A Model containing video metadata such as duration, resolution, frame rate, and codec details.

Source code in datachain/lib/file.py
1321
1322
1323
1324
1325
1326
1327
1328
1329
1330
1331
1332
1333
1334
1335
1336
1337
1338
1339
1340
1341
def get_info(self, video_stream_index: int = 0) -> "Video":
    """
    Retrieves metadata and information about the video file.

    With prefetch and cache off, reads only the parts of the file it needs.

    Args:
        video_stream_index: Zero-based index among video streams to inspect.
            Defaults to 0.

    Returns:
        Video: A Model containing video metadata such as duration,
               resolution, frame rate, and codec details.
    """
    from .video import video_info

    if video_stream_index not in self._video_info_cache:
        self._video_info_cache[video_stream_index] = video_info(
            self, video_stream_index=video_stream_index
        )
    return self._video_info_cache[video_stream_index]

save

save(destination: str, client_config: dict | None = None, *, content_type: str | None = None, content_disposition: str | None = None, cache_control: str | None = None, content_encoding: str | None = None, metadata: dict[str, str] | None = None, write_options: dict[str, Any] | None = None) -> VideoFile

Writes its content to destination

Source code in datachain/lib/file.py
1461
1462
1463
1464
1465
1466
1467
1468
1469
1470
1471
1472
1473
1474
1475
1476
1477
1478
1479
1480
1481
1482
1483
1484
1485
1486
def save(  # type: ignore[override]
    self,
    destination: str,
    client_config: dict | None = None,
    *,
    content_type: str | None = None,
    content_disposition: str | None = None,
    cache_control: str | None = None,
    content_encoding: str | None = None,
    metadata: dict[str, str] | None = None,
    write_options: dict[str, Any] | None = None,
) -> "VideoFile":
    """Writes its content to destination"""
    result = super().save(
        destination,
        client_config=client_config,
        content_type=content_type,
        content_disposition=content_disposition,
        cache_control=cache_control,
        content_encoding=content_encoding,
        metadata=metadata,
        write_options=write_options,
    )
    vf = VideoFile(**result.model_dump())
    vf._set_stream(self._catalog)
    return vf

VideoFrame

VideoFrame(**kwargs)

Bases: DataModel

A data model for representing a video frame.

This model inherits from the VideoFile model and adds a frame attribute, which represents a specific frame within a video file. It allows access to individual frames and provides functionality for reading and saving video frames as image files.

Attributes:

  • video (VideoFile) –

    The video file containing the video frame.

  • frame (int) –

    The frame number referencing a specific frame in the video file.

  • video_stream_index (int) –

    Zero-based index among video streams containing the frame.

  • timestamp (float) –

    Frame timestamp in seconds. Frames returned by VideoFile.get_frame() use FPS metadata. Frames yielded by VideoFile.get_frames() use decoded frame timestamps when available.

Note

The decoded av.VideoFrame is cached in the _decoded instance attribute the first time any of get_np(), read_bytes(), or save() is called. Frames yielded by VideoFile.get_frames() are pre-populated with the decoded frame, so the first call to any of those methods requires no I/O. The cache is held for the lifetime of the VideoFrame object; when many frames are kept in memory at once (e.g. list(video.get_frames())) the cached pixel buffers can create significant memory pressure (~3 MB each for 1080p video). The cache is excluded from pickling so cross-process serialisation (e.g. DataChain .gen() workers) is unaffected.

Source code in datachain/lib/file.py
1777
1778
1779
def __init__(self, **kwargs):
    super().__init__(**kwargs)
    self._decoded = None

get_np

get_np() -> ndarray

Returns a video frame from the video file as a NumPy array.

The decoded av.VideoFrame is cached in _decoded on the first call, so repeated calls to get_np() return the cached pixels without a second decode pass regardless of whether the frame came from VideoFile.get_frames() or VideoFile.get_frame().

Display rotation is applied (matching FFmpeg/OpenCV), so a 90/270 rotation swaps width and height. Channels are RGB; consumers expecting BGR (e.g. OpenCV, Ultralytics) must convert.

Returns:

  • ndarray ( ndarray ) –

    A NumPy array representing the video frame, in the shape (height, width, channels).

Source code in datachain/lib/file.py
1787
1788
1789
1790
1791
1792
1793
1794
1795
1796
1797
1798
1799
1800
1801
1802
1803
1804
1805
1806
1807
1808
1809
1810
1811
1812
def get_np(self) -> "ndarray":
    """
    Returns a video frame from the video file as a NumPy array.

    The decoded ``av.VideoFrame`` is cached in ``_decoded`` on the first
    call, so repeated calls to ``get_np()`` return the cached pixels
    without a second decode pass regardless of whether the frame came
    from ``VideoFile.get_frames()`` or ``VideoFile.get_frame()``.

    Display rotation is applied (matching FFmpeg/OpenCV), so a 90/270
    rotation swaps width and height. Channels are RGB; consumers expecting
    BGR (e.g. OpenCV, Ultralytics) must convert.

    Returns:
        ndarray: A NumPy array representing the video frame,
                 in the shape (height, width, channels).
    """
    from .video import _decode_video_frame, _frame_to_ndarray

    if self._decoded is None:
        self._decoded, _ = _decode_video_frame(
            self.video,
            self.frame,
            self.video_stream_index,
        )
    return _frame_to_ndarray(self._decoded)

read_bytes

read_bytes(format: str = 'jpg') -> bytes

Returns a video frame from the video file as image bytes.

Uses the cached decoded frame when available (e.g. after get_np() or for frames yielded by VideoFile.get_frames()), avoiding a second decode pass.

Parameters:

  • format (str, default: 'jpg' ) –

    The desired image format (e.g., 'jpg', 'png'). Defaults to 'jpg'.

Returns:

  • bytes ( bytes ) –

    The encoded video frame as image bytes.

Source code in datachain/lib/file.py
1814
1815
1816
1817
1818
1819
1820
1821
1822
1823
1824
1825
1826
1827
1828
1829
1830
1831
1832
1833
1834
1835
def read_bytes(self, format: str = "jpg") -> bytes:
    """
    Returns a video frame from the video file as image bytes.

    Uses the cached decoded frame when available (e.g. after ``get_np()``
    or for frames yielded by ``VideoFile.get_frames()``), avoiding a second
    decode pass.

    Args:
        format (str): The desired image format (e.g., 'jpg', 'png').
                      Defaults to 'jpg'.

    Returns:
        bytes: The encoded video frame as image bytes.
    """
    from PIL import Image as PilImage

    from .video import _image_format

    buf = BytesIO()
    PilImage.fromarray(self.get_np()).save(buf, format=_image_format(format))
    return buf.getvalue()

save

save(destination: str | PathLike[str], format: str = 'jpg', client_config: dict | None = None, *, content_type: str | None = None, content_disposition: str | None = None, cache_control: str | None = None, content_encoding: str | None = None, metadata: dict[str, str] | None = None, write_options: dict[str, Any] | None = None) -> ImageFile

Saves the current video frame as an image file.

Uses the cached decoded frame when available (e.g. after get_np() or for frames yielded by VideoFile.get_frames()), avoiding a second decode pass.

If destination is a remote path, the image file will be uploaded to remote storage.

Parameters:

  • destination (str | PathLike[str]) –

    Output directory path or URI (e.g. s3://…, gs://…).

  • format (str, default: 'jpg' ) –

    Image format (e.g., 'jpg', 'png'). Defaults to 'jpg'.

  • client_config (dict | None, default: None ) –

    Optional client configuration (e.g. credentials).

  • content_type (str | None, default: None ) –

    Content-Type to set on the written object.

  • content_disposition (str | None, default: None ) –

    Content-Disposition to set on the object.

  • cache_control (str | None, default: None ) –

    Cache-Control to set on the object.

  • content_encoding (str | None, default: None ) –

    Content-Encoding to set on the object.

  • metadata (dict[str, str] | None, default: None ) –

    Custom key/value metadata to attach to the object.

  • write_options (dict[str, Any] | None, default: None ) –

    Raw, backend-native write kwargs merged after the normalized fields above.

Returns:

  • ImageFile ( ImageFile ) –

    A Model representing the saved image file.

Source code in datachain/lib/file.py
1837
1838
1839
1840
1841
1842
1843
1844
1845
1846
1847
1848
1849
1850
1851
1852
1853
1854
1855
1856
1857
1858
1859
1860
1861
1862
1863
1864
1865
1866
1867
1868
1869
1870
1871
1872
1873
1874
1875
1876
1877
1878
1879
1880
1881
1882
1883
1884
1885
1886
1887
1888
1889
1890
1891
1892
1893
1894
1895
1896
1897
1898
1899
1900
def save(
    self,
    destination: str | os.PathLike[str],
    format: str = "jpg",
    client_config: dict | None = None,
    *,
    content_type: str | None = None,
    content_disposition: str | None = None,
    cache_control: str | None = None,
    content_encoding: str | None = None,
    metadata: dict[str, str] | None = None,
    write_options: dict[str, Any] | None = None,
) -> "ImageFile":
    """
    Saves the current video frame as an image file.

    Uses the cached decoded frame when available (e.g. after ``get_np()``
    or for frames yielded by ``VideoFile.get_frames()``), avoiding a second
    decode pass.

    If ``destination`` is a remote path, the image file will be uploaded
    to remote storage.

    Args:
        destination: Output directory path or URI (e.g. ``s3://…``, ``gs://…``).
        format: Image format (e.g., 'jpg', 'png'). Defaults to 'jpg'.
        client_config: Optional client configuration (e.g. credentials).
        content_type: ``Content-Type`` to set on the written object.
        content_disposition: ``Content-Disposition`` to set on the object.
        cache_control: ``Cache-Control`` to set on the object.
        content_encoding: ``Content-Encoding`` to set on the object.
        metadata: Custom key/value metadata to attach to the object.
        write_options: Raw, backend-native write kwargs merged after the
            normalized fields above.

    Returns:
        ImageFile: A Model representing the saved image file.
    """
    catalog = self.video._catalog
    if catalog is None:
        raise RuntimeError("Cannot save video frame: catalog is not set")

    destination = stringify_path(destination)
    image_bytes = self.read_bytes(format)
    extension = format.removeprefix(".")
    output_file = posixpath.join(
        destination, f"{self.video.get_file_stem()}_{self.frame:04d}.{extension}"
    )
    client, rel_path = self.video._resolve_destination(output_file, client_config)
    result = client.upload(
        image_bytes,
        rel_path,
        write_config=WriteConfig(
            content_type=content_type,
            content_disposition=content_disposition,
            cache_control=cache_control,
            content_encoding=content_encoding,
            metadata=metadata,
            write_options=write_options,
        ),
    )
    image = ImageFile(**result.model_dump())
    image._set_stream(catalog)
    return image

VideoFragment

Bases: DataModel

A data model for representing a video fragment.

This model inherits from the VideoFile model and adds start and end attributes, which represent a specific fragment within a video file. It allows access to individual fragments and provides functionality for reading and saving video fragments as separate video files.

Attributes:

  • video (VideoFile) –

    The video file containing the video fragment.

  • start (float) –

    The starting time of the video fragment in seconds.

  • end (float) –

    The ending time of the video fragment in seconds.

save

save(destination: str, format: str | None = None, client_config: dict | None = None, timeout: float | None = None, *, content_type: str | None = None, content_disposition: str | None = None, cache_control: str | None = None, content_encoding: str | None = None, metadata: dict[str, str] | None = None, write_options: dict[str, Any] | None = None) -> VideoFile

Saves the video fragment as a new video file.

If destination is a remote path, the video file will be uploaded to remote storage.

The output is named {stem}_{hash}_{start_ms}_{end_ms}.{format}, where hash is derived from the source video's source and path, so videos with the same name in different directories don't overwrite each other.

Parameters:

  • destination (str) –

    Output directory path or URI (e.g. s3://…, gs://…).

  • format (str | None, default: None ) –

    Output video format (e.g., 'mp4', 'avi'). If None, inferred from the file extension.

  • client_config (dict | None, default: None ) –

    Optional client configuration (e.g. credentials).

  • timeout (float | None, default: None ) –

    FFmpeg subprocess timeout in seconds. If None, a timeout is computed from the fragment duration. Set to 0 to disable.

  • content_type (str | None, default: None ) –

    Content-Type to set on the written object.

  • content_disposition (str | None, default: None ) –

    Content-Disposition to set on the object.

  • cache_control (str | None, default: None ) –

    Cache-Control to set on the object.

  • content_encoding (str | None, default: None ) –

    Content-Encoding to set on the object.

  • metadata (dict[str, str] | None, default: None ) –

    Custom key/value metadata to attach to the object.

  • write_options (dict[str, Any] | None, default: None ) –

    Raw, backend-native write kwargs merged after the normalized fields above.

Returns:

  • VideoFile ( VideoFile ) –

    A Model representing the saved video file.

Source code in datachain/lib/file.py
1922
1923
1924
1925
1926
1927
1928
1929
1930
1931
1932
1933
1934
1935
1936
1937
1938
1939
1940
1941
1942
1943
1944
1945
1946
1947
1948
1949
1950
1951
1952
1953
1954
1955
1956
1957
1958
1959
1960
1961
1962
1963
1964
1965
1966
1967
1968
1969
1970
1971
1972
1973
1974
1975
1976
1977
1978
1979
1980
1981
1982
1983
def save(
    self,
    destination: str,
    format: str | None = None,
    client_config: dict | None = None,
    timeout: float | None = None,
    *,
    content_type: str | None = None,
    content_disposition: str | None = None,
    cache_control: str | None = None,
    content_encoding: str | None = None,
    metadata: dict[str, str] | None = None,
    write_options: dict[str, Any] | None = None,
) -> "VideoFile":
    """
    Saves the video fragment as a new video file.

    If ``destination`` is a remote path, the video file will be uploaded
    to remote storage.

    The output is named ``{stem}_{hash}_{start_ms}_{end_ms}.{format}``, where
    ``hash`` is derived from the source video's ``source`` and ``path``, so
    videos with the same name in different directories don't overwrite each
    other.

    Args:
        destination: Output directory path or URI (e.g. ``s3://…``, ``gs://…``).
        format: Output video format (e.g., 'mp4', 'avi').
                If None, inferred from the file extension.
        client_config: Optional client configuration (e.g. credentials).
        timeout: FFmpeg subprocess timeout in seconds. If None, a timeout is
            computed from the fragment duration. Set to 0 to disable.
        content_type: ``Content-Type`` to set on the written object.
        content_disposition: ``Content-Disposition`` to set on the object.
        cache_control: ``Cache-Control`` to set on the object.
        content_encoding: ``Content-Encoding`` to set on the object.
        metadata: Custom key/value metadata to attach to the object.
        write_options: Raw, backend-native write kwargs merged after the
            normalized fields above.

    Returns:
        VideoFile: A Model representing the saved video file.
    """
    from .video import save_video_fragment

    return save_video_fragment(
        self.video,
        self.start,
        self.end,
        destination,
        format,
        client_config=client_config,
        timeout=timeout,
        write_config=WriteConfig(
            content_type=content_type,
            content_disposition=content_disposition,
            cache_control=cache_control,
            content_encoding=content_encoding,
            metadata=metadata,
            write_options=write_options,
        ),
    )

Video

Bases: DataModel

A data model representing metadata for a video file.

Attributes:

  • width (int) –

    Display width in pixels (after stream rotation). Defaults to -1 if unknown.

  • height (int) –

    Display height in pixels (after stream rotation). Defaults to -1 if unknown.

  • fps (float) –

    The frame rate of the video (frames per second). Defaults to -1.0 if unknown.

  • duration (float) –

    The total duration of the video in seconds. Defaults to -1.0 if unknown.

  • frames (int) –

    The total number of frames in the video. Defaults to -1 if unknown.

  • format (str) –

    The format of the video file (e.g., 'mp4', 'avi'). Defaults to an empty string.

  • codec (str) –

    The codec used for encoding the video. Defaults to an empty string.