Core I/O and media
Logical paths, local and remote URIs, serialization, image/video normalization, and tiled media processing.
On this page
Core I/O gives model integrations one contract for where data lives and how it is represented. Path helpers resolve logical WorldFoundry locations; storage helpers operate on local paths and supported URI schemes; serialization chooses a format explicitly or from a suffix; media helpers normalize common image/video inputs before model-specific preprocessing begins.
Resolve logical paths, do not hard-code hosts
worldfoundry_path_tokens computes roots for checkpoints, datasets, models, artifacts, caches, source repositories, and Conda environments. Explicit environment values win; otherwise the resolver uses predictable WorldFoundry cache or repository-adjacent defaults.
from worldfoundry.core.io.paths import (
checkpoint_root_path,
resolve_worldfoundry_path,
worldfoundry_path_tokens,
)
env = {
"WORLDFOUNDRY_HOME": "/srv/wf",
"WORLDFOUNDRY_CKPT_DIR": "/models/checkpoints",
}
tokens = worldfoundry_path_tokens(env)
assert tokens["WORLDFOUNDRY_CKPT_DIR"] == "/models/checkpoints"
assert checkpoint_root_path("matrix-game-2", env=env) == \
resolve_worldfoundry_path("${WORLDFOUNDRY_CKPT_DIR}/matrix-game-2", env)Passing an env mapping makes path resolution testable without mutating the process environment. resolve_data_path is different: it points inside package-owned static data and should not be used for downloaded datasets.
Serialization round trip
from pathlib import Path
from tempfile import TemporaryDirectory
from worldfoundry.core import dump_serialized, load_serialized
with TemporaryDirectory() as directory:
path = Path(directory) / "request.yaml"
dump_serialized({"seed": 42, "actions": ["forward", "left"]}, path)
payload = load_serialized(path)
assert payload["seed"] == 42When file_format is omitted, the suffix selects JSON, YAML, JSONL, pickle/gzip, NumPy, Torch, image, video, CSV/Pandas, or tar handling. When no file is supplied, text and binary formats return their serialized value. Pickle and unrestricted Torch checkpoints are executable formats; do not load them from an untrusted source.
Video shape boundary
coerce_video_frames is the normalization boundary for paths, Torch tensors, NumPy arrays, PIL frame lists, and tensor frame lists. It returns a uint8 NumPy array in T × H × W × C layout. video_tensor_to_uint8_frames handles normalized C × T × H × W or single-batch tensors and makes the value range explicit. read_video adds decoder metadata, while load_frames_from_video is for selected frame indices.
Use materialize_video_input when a subprocess or external runtime requires a local filename. Use TileProcessor only for codec/model functions that explicitly support overlapping spatiotemporal tiles; it changes execution layout but blends overlaps back into one output.
Complete reference
The blocks below are the generated signatures for this category. Use the on-page symbol index to jump; source links open the defining implementation behind each lazy export.
73 public symbols
checkpoint_root_path
def checkpoint_root_path(*parts: str | Path,specific_env: str | None = None,env: Mapping[str, str] | None = None) -> PathOverview
Resolves a target model checkpoint directory path with nested subfolders. Belongs to Core I/O and media (paths, URIs, images, video, serialization). Annotated return type: Path.
Source docstring
Resolves a target model checkpoint directory path with nested subfolders.
Avoids host-specific hardcoding by querying general and model-specific variables.
Parameters
partsstr | Pathspecific_envstr | Nonedefault NoneWORLDFOUNDRY_CKPT_DIR`.envMapping[str, str] | Nonedefault Noneos.environ`.Returns
coerce_video_frames
def coerce_video_frames(video_input,max_frames: int | None = None,from_end: bool = False)Overview
Normalize video inputs to uint8 THWC, optionally keeping a prefix/tail. Belongs to Core I/O and media (paths, URIs, images, video, serialization).
Source docstring
Normalize video inputs to uint8 THWC, optionally keeping a prefix/tail.
File inputs use the same ImageIO decoder as the full reader. Prefix reads stop early; tail reads retain only the requested frame count while reading to EOF. Array/tensor inputs retain whole-input range detection before slicing, so truncation does not change automatic value normalization.
Parameters
video_inputmax_framesint | Nonedefault Nonefrom_endbooldefault Falseclass Color()Small color helper compatible with legacy model-runtime config printers. Belongs to Core I/O and media (paths, URIs, images, video, serialization).
Methods
def copy_uri(src: str | os.PathLike[str],dst: str | os.PathLike[str],**storage_options) -> strOverview
Copy one URI to another using storage-aware byte streams. Belongs to Core I/O and media (paths, URIs, images, video, serialization). Annotated return type: str.
Source docstring
Copy one URI to another using storage-aware byte streams.
Remote transfers stream in bounded chunks instead of buffering the whole object in memory; this is the byte mover underneath the download caches, where objects are multi-GB checkpoints.
Parameters
srcstr | os.PathLike[str]dststr | os.PathLike[str]storage_optionsReturns
crop_and_resize
def crop_and_resize(image: Image.Image,height: int,width: int) -> Image.ImageCenter-crop and resize a PIL image to `(width, height). Belongs to Core I/O and media (paths, URIs, images, video, serialization). Annotated return type: Image.Image`.
Parameters
imageImage.ImageheightintwidthintReturns
dump_serialized
def dump_serialized(obj: Any,file: str | Path | IO[Any] | None = None,file_format: str | None = None,encoding: str = 'utf-8',**kwargs: Any) -> str | bytes | NoneOverview
Dump an object to a URI, file object, or string when `file is None. Belongs to Core I/O and media (paths, URIs, images, video, serialization). Annotated return type: str | bytes | None`.
Parameters
objAnyfilestr | Path | IO[Any] | Nonedefault Nonefile_formatstr | Nonedefault Noneencodingstrdefault 'utf-8'kwargsAnyReturns
exists_uri
def exists_uri(uri: str | os.PathLike[str],**storage_options) -> boolReturn whether a WorldFoundry URI (local or remote scheme) currently exists.
Parameters
uristr | os.PathLike[str]storage_optionsReturns
def extract_frames_from_video_url(video_url: str)Decode a remote video URL and return RGB PIL frames. Belongs to Core I/O and media (paths, URIs, images, video, serialization).
Parameters
video_urlstrdef extract_video_frames_to_directory(video_path: str | Path,output_dir: str | Path,prefix: str = 'frames_',suffix: str = '.png',ffmpeg_path: str | Path | None = None,threads: int = 1,timeout_seconds: float | None = None,overwrite: bool = False) -> tuple[Path, ...]Overview
Extract every source frame with bounded, deterministic ffmpeg settings. Belongs to Core I/O and media (paths, URIs, images, video, serialization). Annotated return type: tuple[Path, ...].
Parameters
video_pathstr | Pathoutput_dirstr | Pathprefixstrdefault 'frames_'suffixstrdefault '.png'ffmpeg_pathstr | Path | Nonedefault Nonethreadsintdefault 1timeout_secondsfloat | Nonedefault Noneoverwritebooldefault FalseReturns
file_sha256
def file_sha256(path: str | os.PathLike[str],chunk_size: int = 1024 * 1024) -> strOverview
Compute a file's SHA-256 digest without loading the whole artifact into memory. Belongs to Core I/O and media (paths, URIs, images, video, serialization). Annotated return type: str.
Parameters
pathstr | os.PathLike[str]chunk_sizeintdefault 1024 * 1024Returns
get_video_details
def get_video_details(video_path: str | Path) -> tuple[int, float, float]Overview
Return `(total_frames, fps, duration_seconds) for a local video. Belongs to Core I/O and media (paths, URIs, images, video, serialization). Annotated return type: tuple[int, float, float]`.
Parameters
video_pathstr | PathReturns
hf_download_or_fpath
def hf_download_or_fpath(path: str | PathLike[str] | None) -> str | AnyBackwards-compatible alias for :func:resolve_hf_path. Belongs to Core I/O and media (paths, URIs, images, video, serialization). Annotated return type: str | Any.
Parameters
pathstr | PathLike[str] | NoneReturns
hfd_root_path
def hfd_root_path(*parts: str | Path,env: Mapping[str, str] | None = None) -> PathResolves the hfd-style local downloader checkpoint directory. Belongs to Core I/O and media (paths, URIs, images, video, serialization). Annotated return type: Path.
Parameters
partsstr | PathenvMapping[str, str] | Nonedefault NoneReturns
class IdentityVideoPostProcessor()Zero-copy processor used to exercise the production integration path. Belongs to Core I/O and media (paths, URIs, images, video, serialization).
Methods
Preset id used by Studio when no real processor is selected.
Parameters
selfReturns
infer_serialization_format
def infer_serialization_format(file: str | Path | IO[Any] | None,file_format: str | None = None) -> strInfer a normalized serialization format key. Belongs to Core I/O and media (paths, URIs, images, video, serialization). Annotated return type: str.
Parameters
filestr | Path | IO[Any] | Nonefile_formatstr | Nonedefault NoneReturns
inspect_rtx_vfx_capability
def inspect_rtx_vfx_capability(, device: int = 0) -> RTXVFXCapabilityInspect static prerequisites without creating a VFX effect. Belongs to Core I/O and media (paths, URIs, images, video, serialization). Annotated return type: RTXVFXCapability.
Parameters
deviceintdefault 0Returns
is_remote_uri
def is_remote_uri(uri: str | os.PathLike[str]) -> boolReturn whether a path needs a non-local storage backend. Belongs to Core I/O and media (paths, URIs, images, video, serialization). Annotated return type: bool.
Parameters
uristr | os.PathLike[str]Returns
def join_uri(base: str | os.PathLike[str],*parts: str | os.PathLike[str]) -> strJoin path components without losing remote URI schemes. Belongs to Core I/O and media (paths, URIs, images, video, serialization). Annotated return type: str.
Parameters
basestr | os.PathLike[str]partsstr | os.PathLike[str]Returns
def jsonable(value: Any) -> AnyRecursively convert common runtime objects into JSON-safe values. Belongs to Core I/O and media (paths, URIs, images, video, serialization). Annotated return type: Any.
Parameters
valueAnyto_dict object, path, mapping, sequence, tensor/ array-like object with tolist`, callable, primitive, or fallback object.Returns
Notes
Mapping keys are stringified, sets become lists, callables become a module/qualified-name record, and otherwise unsupported objects fall back to `repr`. This is evidence serialization, not a reversible object codec.
load_frames_from_video
def load_frames_from_video(video_path: str | Path,indices: Iterable[int],video_decode_backend: str = 'decord',eval_: bool = True)Load selected RGB frames into a torch tensor using decord or OpenCV. Belongs to Core I/O and media (paths, URIs, images, video, serialization).
Parameters
video_pathstr | PathindicesIterable[int]video_decode_backendstrdefault 'decord'eval_booldefault Trueload_serialized
def load_serialized(file: str | Path | IO[Any],file_format: str | None = None,encoding: str = 'utf-8',allow_pickle: bool = False,**kwargs: Any) -> AnyOverview
Load a structured object from a URI or file object. Belongs to Core I/O and media (paths, URIs, images, video, serialization). Annotated return type: Any.
Source docstring
Load a structured object from a URI or file object.
Pickle and gzip-pickle payloads are rejected by default because unpickling executes arbitrary code. Trusted callers must pass `allow_pickle=True. Torch formats independently default to weights_only=True. .gz` is treated as gzip-compressed pickle for backward compatibility; use an explicit non-gzip format for other compressed data.
Parameters
filestr | Path | IO[Any]file_formatstr | Nonedefault Noneencodingstrdefault 'utf-8'allow_picklebooldefault FalsekwargsAnyReturns
load_video_frames
def load_video_frames(video_path: str | Path,max_frames: int | None = None,from_end: bool = False)Decode a video, optionally bounding its prefix/tail without changing RGB conversion. Belongs to Core I/O and media (paths, URIs, images, video, serialization).
Parameters
video_pathstr | Pathmax_framesint | Nonedefault Nonefrom_endbooldefault Falselocal_model_root_path
def local_model_root_path(env: Mapping[str, str] | None = None) -> PathOverview
Resolves the root directory containing local model weights, configs, and adapters. Belongs to Core I/O and media (paths, URIs, images, video, serialization). Annotated return type: Path.
Parameters
envMapping[str, str] | Nonedefault NoneReturns
class LowMemoryImageFolder(folder,file_list = None)Lazy image-folder reader that loads frames on demand. Belongs to Core I/O and media (paths, URIs, images, video, serialization).
Parameters
folderfile_listdefault NoneLowMemoryVideo
class LowMemoryVideo(file_name: str)Lazy video reader that loads frames on demand. Belongs to Core I/O and media (paths, URIs, images, video, serialization).
Parameters
file_namestrmaterialize_hf_snapshot
def materialize_hf_snapshot(repo_id_or_path: str | os.PathLike[str],revision: str | None = None,cache_dir: str | os.PathLike[str] | None = None,allow_patterns: str | Sequence[str] | None = None,ignore_patterns: str | Sequence[str] | None = None,required_files: Sequence[str] = (),local_files_only: bool | None = None,token: str | bool | None = None) -> PathOverview
Return a local snapshot for either a path or Hugging Face repo id. Belongs to Core I/O and media (paths, URIs, images, video, serialization). Annotated return type: Path.
Source docstring
Return a local snapshot for either a path or Hugging Face repo id.
Remote downloads are serialized on rank zero and then reopened in local-only mode, avoiding partial-cache races in multi-GPU jobs.
Parameters
repo_id_or_pathstr | os.PathLike[str]revisionstr | Nonedefault Nonecache_dirstr | os.PathLike[str] | Nonedefault Noneallow_patternsstr | Sequence[str] | Nonedefault Noneignore_patternsstr | Sequence[str] | Nonedefault Nonerequired_filesSequence[str]default ()local_files_onlybool | Nonedefault Nonetokenstr | bool | Nonedefault NoneReturns
materialize_video_input
def materialize_video_input(video_input,output_dir: Optional[str] = None,filename: str = 'input.mp4',fps: int = 24) -> strReturn a local video path, materializing in-memory inputs when needed. Belongs to Core I/O and media (paths, URIs, images, video, serialization). Annotated return type: str.
Parameters
video_inputoutput_dirOptional[str]default Nonefilenamestrdefault 'input.mp4'fpsintdefault 24Returns
def maybe_download_hf_repo_on_rank0(repo_id_or_path: str,revision: str | None = None,cache_dir: str | os.PathLike[str] | None = None,allow_patterns: str | Sequence[str] | None = None,ignore_patterns: str | Sequence[str] | None = None,token: str | bool | None = None) -> NoneOverview
Download a remote HF repo snapshot from rank 0 when downloads are allowed. Belongs to Core I/O and media (paths, URIs, images, video, serialization). Annotated return type: None.
Source docstring
Download a remote HF repo snapshot from rank 0 when downloads are allowed.
Local paths and explicit offline/local-only modes are no-ops. For remote repositories, rank 0 preloads the snapshot while other distributed ranks wait for its success/failure signal. A filesystem lock serializes independent processes that share the same HF cache directory.
Parameters
repo_id_or_pathstrrevisionstr | Nonedefault Nonecache_dirstr | os.PathLike[str] | Nonedefault Noneallow_patternsstr | Sequence[str] | Nonedefault Noneignore_patternsstr | Sequence[str] | Nonedefault Nonetokenstr | bool | Nonedefault NoneReturns
merge_video_audio
def merge_video_audio(video_path: str,audio_path: str) -> NoneOverview
Merge video and audio with ffmpeg; overwrite `video_path on success. Belongs to Core I/O and media (paths, URIs, images, video, serialization). Annotated return type: None`.
Source docstring
Merge video and audio with ffmpeg; overwrite `video_path` on success.
Raises on failure (missing inputs, ffmpeg errors). The temporary mux output is always removed, so a failed merge leaves the original video untouched. Historically this function swallowed its own errors and returned normally, which made callers treat missing audio tracks as success.
Parameters
video_pathstraudio_pathstrReturns
def open_uri(uri: str | os.PathLike[str],mode: str = 'rb',encoding: str = 'utf-8',**storage_options) -> Generator[BinaryIO | TextIO, None, None]Overview
Open a local or remote URI. Belongs to Core I/O and media (paths, URIs, images, video, serialization). Annotated return type: Generator[BinaryIO | TextIO, None, None].
Source docstring
Open a local or remote URI.
`fsspec` is used when available. Without it, local files are supported for all modes and HTTP(S) URLs are supported for reads.
Parameters
uristr | os.PathLike[str]modestrdefault 'rb'encodingstrdefault 'utf-8'storage_optionsReturns
package_module_root
def package_module_root(package: str) -> PathResolve the source directory for an importable package. Belongs to Core I/O and media (paths, URIs, images, video, serialization). Annotated return type: Path.
Parameters
packagestrReturns
ParallelHelper
class ParallelHelper()Overview
Distribute video tiles by cost and reconstruct their global ordering. Belongs to Core I/O and media (paths, URIs, images, video, serialization).
Source docstring
Distribute video tiles by cost and reconstruct their global ordering.
In non-distributed execution every tile remains local. With a process group, larger tiles are round-robin balanced across ranks and decoded frame tensors are gathered with explicit count, dtype, and shape metadata.
Methods
split_tile_list(tile_numel_dict: OrderedDict[int, int],parallel_group: torch.distributed.ProcessGroup = None) -> List[int]SourceOverview
Splits the given tile size into a list of sizes that each rank should handle.
Source docstring
Splits the given tile size into a list of sizes that each rank should handle.
This method takes into account the number of ranks in a distributed setting. If the distributed environment is not initialized, it returns a list of integers from 0 to tile_size - 1, representing each tile index.
If the distributed environment is initialized, it calculates the base tile size for each rank and distributes any remaining tiles among the ranks.
Parameters
tile_numel_dictOrderedDict[int, int]parallel_grouptorch.distributed.ProcessGroupdefault NoneReturns
gather_frames(frames: List[torch.Tensor],global_tile_idxs: List[int],parallel_group: torch.distributed.ProcessGroup = None) -> List[torch.Tensor]SourceOverview
Gathers frame data from all ranks in a distributed environment.
Source docstring
Gathers frame data from all ranks in a distributed environment.
This method collects frames from all ranks and combines them into a single list. If the distributed environment is not initialized, it simply returns the input frames.
Parameters
framesList[torch.Tensor]global_tile_idxsList[int]parallel_grouptorch.distributed.ProcessGroupdefault NoneReturns
Overview
Converts a single index into a list of indices, representing the position in a multi-dimensional space.
Source docstring
Converts a single index into a list of indices, representing the position in a multi-dimensional space.
This method takes an integer index and a list of loop sizes, and converts the index into a list of indices that correspond to the position in a multi-dimensional space.
Parameters
indexintloop_sizeList[int]Returns
Overview
Converts a list of indices into a single index, representing the position in a multi-dimensional space.
Source docstring
Converts a list of indices into a single index, representing the position in a multi-dimensional space.
This method takes a list of indices and a list of loop sizes, and converts the list of indices into a single index that corresponds to the position in a multi-dimensional space.
Parameters
indexList[int]loop_sizeList[int]Returns
parse_uri_scheme
def parse_uri_scheme(uri: str | os.PathLike[str]) -> strReturn the lowercase URI scheme, using `file for local paths. Belongs to Core I/O and media (paths, URIs, images, video, serialization). Annotated return type: str`.
Parameters
uristr | os.PathLike[str]Returns
probe_rtx_vfx_runtime
def probe_rtx_vfx_runtime(device: int = 0,config: RTXVideoSuperResolutionConfig | None = None,input_spec: VideoSpec | None = None,run_inference: bool = True) -> RTXVFXCapabilityOverview
Load the native effect and optionally process one representative frame. Belongs to Core I/O and media (paths, URIs, images, video, serialization). Annotated return type: RTXVFXCapability.
Parameters
deviceintdefault 0run_inferencebooldefault TrueReturns
project_root
def project_root(start: str | Path | None = None) -> PathOverview
Walks upward from a starting path to locate the root repository containing pyproject.toml. Belongs to Core I/O and media (paths, URIs, images, video, serialization). Annotated return type: Path.
Source docstring
Walks upward from a starting path to locate the root repository containing pyproject.toml.
This helper provides robust local development support, falling back to a package-relative root if executed from a system-wide python site-packages deployment.
Parameters
startstr | Path | Nonedefault NoneNone` starts from this module's installed source path.Returns
pyproject.toml`, or the package-relative fallback when no repository marker is found.read_binary_uri
def read_binary_uri(uri: str | os.PathLike[str],**storage_options) -> bytesRead all bytes from a URI through Core’s storage helpers.
Parameters
uristr | os.PathLike[str]storage_optionsReturns
read_image_as_video_tensor
def read_image_as_video_tensor(image_path: str | Path,resolution: Iterable[int],num_video_frames: int,resize: bool = True)Load an image and materialize a `[1,C,T,H,W]` uint8 conditioning video tensor. Belongs to Core I/O and media (paths, URIs, images, video, serialization).
Parameters
image_pathstr | PathresolutionIterable[int]num_video_framesintresizebooldefault Trueread_text_uri
def read_text_uri(uri: str | os.PathLike[str],encoding: str = 'utf-8',**storage_options) -> strRead all text from a URI through Core’s storage helpers.
Parameters
uristr | os.PathLike[str]encodingstrdefault 'utf-8'storage_optionsReturns
read_video
def read_video(video_path: str | Path,return_metadata: bool = True)Decode a video into frames and optional metadata. Belongs to Core I/O and media (paths, URIs, images, video, serialization).
Parameters
video_pathstr | Pathreturn_metadatabooldefault Trueread_video_window_rgb
def read_video_window_rgb(video_path: str | Path,start_frame: int = 0,frame_count: int)Overview
Decode up to `frame_count` RGB frames without decoding the tail. Belongs to Core I/O and media (paths, URIs, images, video, serialization).
Source docstring
Decode up to `frame_count` RGB frames without decoding the tail.
Frame offsets are decoded sequentially from the beginning, preserving PyAV/torchvision frame ordering and RGB conversion, including variable frame-rate inputs. A window that reaches EOF returns its available frames; an empty window raises `ValueError`. The decoder and any temporary local copy of a remote input are closed even when decoding fails.
Parameters
video_pathstr | Pathstart_frameintdefault 0frame_countintrequire_rtx_vfx_runtime
def require_rtx_vfx_runtime(device: int = 0,config: RTXVideoSuperResolutionConfig | None = None,input_spec: VideoSpec | None = None,run_inference: bool = True) -> RTXVFXCapabilityOverview
Fully probe the selected RTX runtime and fail with an actionable message. Belongs to Core I/O and media (paths, URIs, images, video, serialization). Annotated return type: RTXVFXCapability.
Parameters
deviceintdefault 0run_inferencebooldefault TrueReturns
def resize_video_tensor_to_resolution(video_tensor,resolution: Iterable[int])Resize and center-crop a `[T,C,H,W] tensor to (target_h, target_w)`. Belongs to Core I/O and media (paths, URIs, images, video, serialization).
Parameters
video_tensorresolutionIterable[int]resolve_data_path
def resolve_data_path(parts: str | Path) -> PathResolves a subpath under the internal worldfoundry/data asset directory. Belongs to Core I/O and media (paths, URIs, images, video, serialization). Annotated return type: Path.
Parameters
partsstr | PathReturns
resolve_hf_path
def resolve_hf_path(path: str | PathLike[str] | None) -> str | AnyOverview
Resolve a possibly `hf://-prefixed path to a local filesystem path. Belongs to Core I/O and media (paths, URIs, images, video, serialization). Annotated return type: str | Any`.
Source docstring
Resolve a possibly `hf://`-prefixed path to a local filesystem path.
Accepts either:
* a local path (returned unchanged if it exists), or * `hf://<owner>/<repo>[/<subpath>]` — reuses an already materialized WorldFoundry-local snapshot, otherwise downloads only the requested subtree through the configured Hugging Face endpoint.
Set `HF_HUB_OFFLINE=1` when runtime I/O must remain strictly offline.
Parameters
pathstr | PathLike[str] | NoneReturns
resolve_hf_snapshot_path
def resolve_hf_snapshot_path(value: str | os.PathLike[str],required_files: Sequence[str] = (),local_files_only_env: str = 'WORLDFOUNDRY_HF_LOCAL_FILES_ONLY',local_files_only: bool | None = None) -> PathResolve a repo id, HF cache repo root, or local path to a usable snapshot. Belongs to Core I/O and media (paths, URIs, images, video, serialization). Annotated return type: Path.
Parameters
valuestr | os.PathLike[str]required_filesSequence[str]default ()local_files_only_envstrdefault 'WORLDFOUNDRY_HF_LOCAL_FILES_ONLY'local_files_onlybool | Nonedefault NoneReturns
def resolve_local_checkpoint_file(model_id_or_path: str | Path,filename: str,env: Mapping[str, str] | None = None) -> PathOverview
Resolve one checkpoint file from explicit or WorldFoundry-local storage. Belongs to Core I/O and media (paths, URIs, images, video, serialization). Annotated return type: Path.
Source docstring
Resolve one checkpoint file from explicit or WorldFoundry-local storage.
An explicit file path is accepted directly. Directory paths and repository identifiers are resolved through :func:resolve_local_hf_model_path, so incomplete hfd transfers are rejected and this function never contacts a model hub.
Parameters
model_id_or_pathstr | PathfilenamestrenvMapping[str, str] | Nonedefault NoneReturns
def resolve_local_hf_model_path(model_id_or_path: str | Path,required_files: Sequence[str] = (),revision: str | None = None,env: Mapping[str, str] | None = None) -> PathOverview
Resolve a Hugging Face model strictly from WorldFoundry-local storage. Belongs to Core I/O and media (paths, URIs, images, video, serialization). Annotated return type: Path.
Source docstring
Resolve a Hugging Face model strictly from WorldFoundry-local storage.
The resolver understands direct export directories, hfd-style names such as `owner--repo, and the Hub cache snapshots/<revision>` layout. It never contacts the network, which keeps model initialization deterministic and works in inference environments that do not install PyTorch.
Parameters
model_id_or_pathstr | Pathrequired_filesSequence[str]default ()revisionstr | Nonedefault NoneenvMapping[str, str] | Nonedefault NoneReturns
resolve_worldfoundry_path
def resolve_worldfoundry_path(value: str | Path,env: Mapping[str, str] | None = None) -> PathOverview
Expands structural WorldFoundry path tokens (e.g. Belongs to Core I/O and media (paths, URIs, images, video, serialization). Annotated return type: Path.
Source docstring
Expands structural WorldFoundry path tokens (e.g. ${WORLDFOUNDRY_CKPT_DIR}) and home markers (~).
Performs precise regex-free variable mapping replacement while preserving subfolder hierarchies.
Parameters
valuestr | Path$NAME or ${NAME}` tokens.envMapping[str, str] | Nonedefault NoneReturns
Path`. The target is not created or required to exist.def rtx_postprocessor_from_preset(preset: str,scale: float | None = None,output_width: int | None = None,output_height: int | None = None,quality: RTXVideoSuperResolutionQuality | None = None,device: int | None = None,non_blocking: bool | None = None,use_current_stream: bool | None = None) -> RTXVideoSuperResolutionPostProcessorOverview
Resolve a stable named preset, then apply explicit operator overrides. Belongs to Core I/O and media (paths, URIs, images, video, serialization). Annotated return type: RTXVideoSuperResolutionPostProcessor.
Parameters
presetstrscalefloat | Nonedefault Noneoutput_widthint | Nonedefault Noneoutput_heightint | Nonedefault NonequalityRTXVideoSuperResolutionQuality | Nonedefault Nonedeviceint | Nonedefault Nonenon_blockingbool | Nonedefault Noneuse_current_streambool | Nonedefault Noneclass RTXVFXCapability(available: bool,device: int,reason: str,package_version: str | None = None,gpu_name: str | None = None,compute_capability: tuple[int, int] | None = None,driver_library: str | None = None,probe_status: RTXVFXProbeStatus = 'not-run',backend_error: str | None = None)Read-only diagnosis for the optional proprietary runtime. Belongs to Core I/O and media (paths, URIs, images, video, serialization).
Attributes
availablebooldeviceintreasonstrpackage_versionstr | Nonedefault Nonegpu_namestr | Nonedefault Nonecompute_capabilitytuple[int, int] | Nonedefault Nonedriver_librarystr | Nonedefault Noneprobe_statusRTXVFXProbeStatusdefault 'not-run'backend_errorstr | Nonedefault NoneMethods
JSON-safe diagnosis; `compute_capability` is a list so manifests need no encoder.
Returns
class RTXVideoSuperResolutionConfig(scale: float = 2.0,output_width: int | None = None,output_height: int | None = None,quality: RTXVideoSuperResolutionQuality = 'HIGH',device: int = 0,clamp_input: bool = True,non_blocking: bool = False,use_current_stream: bool = True,float_range: RTXVideoFloatRange = 'minus-one-one')Overview
Configuration for one RTX VSR processor factory. Belongs to Core I/O and media (paths, URIs, images, video, serialization).
Source docstring
Configuration for one RTX VSR processor factory.
Floating-point input is interpreted using `float_range and emitted in the same range and dtype. uint8 input is interpreted and emitted as display-ready RGB in [0, 255]`.
Attributes
scalefloatdefault 2.0output_widthint | Nonedefault Noneoutput_heightint | Nonedefault NonequalityRTXVideoSuperResolutionQualitydefault 'HIGH'deviceintdefault 0clamp_inputbooldefault Truenon_blockingbooldefault Falseuse_current_streambooldefault Truefloat_rangeRTXVideoFloatRangedefault 'minus-one-one'Methods
Return the layout-independent stream specification emitted by VSR.
Parameters
input_specVideoSpecReturns
Overview
Resolve explicit size or `scale`; same-resolution qualities cannot resize.
Source docstring
Resolve explicit size or `scale`; same-resolution qualities cannot resize.
Standard / high-bitrate modes refuse downscale and cap at 4x so a mis-set Studio override cannot ask VFX for an unsupported geometry.
Parameters
input_specVideoSpecReturns
class RTXVideoSuperResolutionPostProcessor(config: RTXVideoSuperResolutionConfig | None = None,**overrides: Any)Preserve WorldFoundry video contracts while applying NVIDIA RTX VSR. Belongs to Core I/O and media (paths, URIs, images, video, serialization).
Parameters
overridesAnyMethods
Stable chain id; presets share this name so telemetry stays one series.
Parameters
selfReturns
Use the configured VFX device instead of the caller's current GPU.
Parameters
selfReturns
Delegate geometry to the config so the chain can size the sink first.
Parameters
input_specVideoSpecReturns
save_frames
def save_frames(frames,save_path)Write a sequence of PIL frames to numbered PNG files. Belongs to Core I/O and media (paths, URIs, images, video, serialization).
Parameters
framessave_pathsave_image_or_video_tensor
def save_image_or_video_tensor(tensor,save_path,fps: int = 24,quality: int | None = None,ffmpeg_params: list[str] | None = None,value_range: str | tuple[float, float] = 'auto',image_format: str = 'JPEG',video_format: str = 'mp4',**kwargs) -> str | NoneOverview
Save a normalized `[C,T,H,W] or [B,C,T,H,W] tensor as image/video. Belongs to Core I/O and media (paths, URIs, images, video, serialization). Annotated return type: str | None`.
Source docstring
Save a normalized `[C,T,H,W] or [B,C,T,H,W]` tensor as image/video.
A single-frame tensor is saved as an image; multi-frame tensors are saved as videos. Local paths and URI-like targets supported by `worldfoundry.core.io` storage helpers are both accepted.
Parameters
tensorsave_pathfpsintdefault 24qualityint | Nonedefault Noneffmpeg_paramslist[str] | Nonedefault Nonevalue_rangestr | tuple[float, float]default 'auto'image_formatstrdefault 'JPEG'video_formatstrdefault 'mp4'kwargsReturns
save_video
def save_video(frames,save_path,fps,quality = 9,ffmpeg_params = None)Write a sequence of PIL frames to a video file. Belongs to Core I/O and media (paths, URIs, images, video, serialization).
Parameters
framessave_pathfpsqualitydefault 9ffmpeg_paramsdefault Nonesave_video_frames
def save_video_frames(video_frames,output_path: str | Path,fps: int = 16,**kwargs) -> NoneWrite a THWC uint8 frame array/list to a video path or URI. Belongs to Core I/O and media (paths, URIs, images, video, serialization). Annotated return type: None.
Parameters
video_framesoutput_pathstr | Pathfpsintdefault 16kwargsReturns
save_video_with_audio
def save_video_with_audio(frames,save_path,audio_path,fps = 16,quality = 9,ffmpeg_params = None)Write PIL frames, then mux an external audio track with ffmpeg. Belongs to Core I/O and media (paths, URIs, images, video, serialization).
Parameters
framessave_pathaudio_pathfpsdefault 16qualitydefault 9ffmpeg_paramsdefault NoneNotes
`merge_video_audio raises on ffmpeg failures after removing its temporary file, so an exception here means save_path` still holds the silent video.
search_for_images
def search_for_images(folder)Overview
Return naturally sorted JPG/PNG paths from one image-sequence folder. Belongs to Core I/O and media (paths, URIs, images, video, serialization).
Source docstring
Return naturally sorted JPG/PNG paths from one image-sequence folder.
Embedded digit runs are compared numerically, so `2.png sorts before 10.png`. The search is non-recursive.
Parameters
folderTileProcessor
class TileProcessor(encode_fn,decode_fn,tile_sample_min_height: int = 256,tile_sample_min_width: int = 256,tile_sample_min_length: int = 16,spatial_downsample_factor: int = 8,temporal_downsample_factor: int = 1,spatial_tile_overlap_factor: float = 0.25,temporal_tile_overlap_factor: float = 0,sr_ratio = 1,first_frame_as_image: bool = False,parallel_group: torch.distributed.ProcessGroup = None)Overview
Encode or decode large videos through overlapping spatiotemporal tiles. Belongs to Core I/O and media (paths, URIs, images, video, serialization).
Source docstring
Encode or decode large videos through overlapping spatiotemporal tiles.
The processor derives latent tile sizes from codec downsample factors, schedules tiles locally or across `parallel_group, and linearly blends temporal/vertical/horizontal overlaps to suppress seams. encode_fn and decode_fn` remain model-owned callables.
Parameters
encode_fndecode_fntile_sample_min_heightintdefault 256tile_sample_min_widthintdefault 256tile_sample_min_lengthintdefault 16spatial_downsample_factorintdefault 8temporal_downsample_factorintdefault 1spatial_tile_overlap_factorfloatdefault 0.25temporal_tile_overlap_factorfloatdefault 0sr_ratiodefault 1first_frame_as_imagebooldefault Falseparallel_grouptorch.distributed.ProcessGroupdefault NoneMethods
Linear-ramp overlap along time so adjacent temporal tiles do not seam.
Parameters
atorch.Tensorbtorch.Tensorblend_extentintReturns
Linear-ramp overlap along height (vertical tile seams).
Parameters
atorch.Tensorbtorch.Tensorblend_extentintReturns
Linear-ramp overlap along width (horizontal tile seams).
Parameters
atorch.Tensorbtorch.Tensorblend_extentintReturns
Encode *x* tile-by-tile and blend overlaps in latent space.
Parameters
xtorch.FloatTensorverbosebooldefault FalseDecode latent tiles and blend overlaps in pixel space.
Parameters
ztorch.FloatTensorverbosebooldefault Falsedef video_tensor_to_uint8_frames(video_tensor,value_range: str | tuple[float, float] = 'auto',max_frames: int | None = None,from_end: bool = False) -> 'object'Overview
Convert a normalized torch video tensor to uint8 THWC frames. Belongs to Core I/O and media (paths, URIs, images, video, serialization). Annotated return type: 'object'.
Source docstring
Convert a normalized torch video tensor to uint8 THWC frames.
`value_range="auto" treats tensors with negative values as [-1, 1] and non-negative floating tensors as [0, 1]`.
Parameters
video_tensorvalue_rangestr | tuple[float, float]default 'auto'max_framesint | Nonedefault Nonefrom_endbooldefault FalseReturns
VideoChunk
class VideoChunk(frames: Any,layout: VideoTensorLayout = 'frame-list',metadata: dict[str, Any] = <dict factory>)One segment exchanged between a generator, processors, and a sink. Belongs to Core I/O and media (paths, URIs, images, video, serialization).
Attributes
framesAnylayoutVideoTensorLayoutdefault 'frame-list'metadatadict[str, Any]default <dict factory>Methods
Count frames from the declared layout; empty chunks are valid (buffering).
Parameters
selfReturns
class VideoData(video_file = None,image_folder = None,height = None,width = None,**kwargs)Lazy video or image-folder dataset with optional resize. Belongs to Core I/O and media (paths, URIs, images, video, serialization).
Parameters
video_filedefault Noneimage_folderdefault Noneheightdefault Nonewidthdefault NonekwargsMethods
Materialize every frame (defeats laziness; used by small debug clips).
Cap or override reported length without truncating the underlying reader.
Parameters
lengthSet the optional crop-resize target; `None` keeps native size.
Parameters
heightwidthReturn `(H, W)`; probes frame 0 when no explicit size was set.
Dump every frame as `{i}.png` (0-based) under *folder*.
Parameters
folderclass VideoPostprocessChain(processors: tuple[VideoPostProcessor, ...] = ())Ordered processor factories applied to each generated video stream. Belongs to Core I/O and media (paths, URIs, images, video, serialization).
Attributes
Methods
Ordered processor ids for telemetry; empty when the chain is a no-op.
Parameters
selfReturns
Compose declared specs left-to-right so the sink can size buffers first.
Parameters
input_specVideoSpecReturns
Open one session per factory, feeding each the previous output spec.
Parameters
input_specVideoSpecReturns
class VideoPostProcessor()Factory for per-stream processor sessions. Belongs to Core I/O and media (paths, URIs, images, video, serialization).
Methods
Stable telemetry id; subclasses override when the class name is not the preset.
Parameters
selfReturns
Declare an explicit GPU device for optional stream profiling.
Parameters
selfReturns
Declare the spec this processor emits; default is identity (no resize).
Parameters
input_specVideoSpecReturns
class VideoPostProcessorSession()State owned by one logical output stream. Belongs to Core I/O and media (paths, URIs, images, video, serialization).
Methods
Consume one chunk and return zero or more ordered output chunks.
Parameters
chunkVideoChunkReturns
Emit buffered tail frames and transition to a closed state.
Returns
class VideoPostprocessStepStats(elapsed_ms: float,input_frames: int,output_frames: int,buffering: bool,gpu_elapsed_ms: float | None = None)Host dispatch, optional CUDA stream time, and frame-count telemetry. Belongs to Core I/O and media (paths, URIs, images, video, serialization).
Attributes
elapsed_msfloatinput_framesintoutput_framesintbufferingboolgpu_elapsed_msfloat | Nonedefault NoneMethods
JSON-safe receipt; `buffering` is True when the stage emitted no frames.
Returns
class VideoPostprocessStream(chain: VideoPostprocessChain | None = None,fps: float | None = None,profile_cuda: bool = False)Own one stateful processor chain across autoregressive chunks. Belongs to Core I/O and media (paths, URIs, images, video, serialization).
Parameters
fpsfloat | Nonedefault Noneprofile_cudabooldefault FalseMethods
Chain ids for Studio overlays; empty when the default identity chain is used.
Parameters
selfReturns
Number of `process calls completed; used as autoregressive_index`.
Parameters
selfReturns
process(frames: Any,layout: VideoTensorLayout,metadata: dict[str, Any] | None = None) -> list[VideoChunk]SourceOverview
Run one AR chunk; the first call freezes the input spec for the stream.
Source docstring
Run one AR chunk; the first call freezes the input spec for the stream.
A later chunk whose inferred spec differs is rejected so a processor cannot silently resize mid-stream. `finish is terminal until :meth:reset`.
Parameters
framesAnylayoutVideoTensorLayoutmetadatadict[str, Any] | Nonedefault NoneReturns
Flush buffered tails once; later calls return an empty list.
Returns
Discard buffered output, release state, and accept a fresh stream.
Returns
class VideoSpec(height: int,width: int,fps: float | None = None,channels: int = 3,dtype: str | None = None)Static dimensions and cadence for one processor stream. Belongs to Core I/O and media (paths, URIs, images, video, serialization).
Attributes
heightintwidthintfpsfloat | Nonedefault Nonechannelsintdefault 3dtypestr | Nonedefault Noneworldfoundry_path_tokens
def worldfoundry_path_tokens(env: Mapping[str, str] | None = None) -> dict[str, str]Overview
Generates the dictionary of logical path-token replacements used across the system. Belongs to Core I/O and media (paths, URIs, images, video, serialization). Annotated return type: dict[str, str].
Source docstring
Generates the dictionary of logical path-token replacements used across the system.
Builds dynamic mappings for artifact, checkpoint, data, conda, and repo paths. Prioritizes explicit environment overrides (such as WORLDFOUNDRY_HOME or WORLDFOUNDRY_CACHE_DIR) and falls back to user-home cache directories when variables are unset.
Parameters
envMapping[str, str] | Nonedefault Noneos.environ`. Passing a mapping makes resolution deterministic in tests.Returns
write_binary_uri
def write_binary_uri(uri: str | os.PathLike[str],data: bytes | bytearray | memoryview | io.BytesIO,**storage_options) -> NoneWrite bytes to a URI, creating local parent directories as needed. Belongs to Core I/O and media (paths, URIs, images, video, serialization). Annotated return type: None.
Parameters
uristr | os.PathLike[str]databytes | bytearray | memoryview | io.BytesIOstorage_optionsReturns
write_text_uri
def write_text_uri(uri: str | os.PathLike[str],data: str,encoding: str = 'utf-8',**storage_options) -> NoneWrite text to a URI, creating local parent directories as needed. Belongs to Core I/O and media (paths, URIs, images, video, serialization). Annotated return type: None.
Parameters
uristr | os.PathLike[str]datastrencodingstrdefault 'utf-8'storage_optionsReturns
write_video
def write_video(video_frames,output_path: str | Path,fps: int = 16,quality: int | None = None,format: str | None = None,**kwargs) -> NoneOverview
Write a THWC video array/list to a local path or remote URI. Belongs to Core I/O and media (paths, URIs, images, video, serialization). Annotated return type: None.
Source docstring
Write a THWC video array/list to a local path or remote URI.
The ordinary local MP4 path streams RGB bytes straight to FFmpeg. This avoids importing and initializing ImageIO's plugin after inference while retaining its effective default x264 CRF (23). Custom ImageIO options, explicit quality settings, non-MP4 containers, and remote targets keep the general ImageIO path.
Parameters
video_framesoutput_pathstr | Pathfpsintdefault 16qualityint | Nonedefault Noneformatstr | Nonedefault NonekwargsReturns
write_video_torchvision
def write_video_torchvision(filename: str | Path,video_array: Any,fps: float,*args: Any,**kwargs: Any) -> NoneOverview
Write RGB video frames with a `torchvision.io.write_video-compatible signature. Belongs to Core I/O and media (paths, URIs, images, video, serialization). Annotated return type: None`.
Parameters
filenamestr | Pathvideo_arrayAnyfpsfloatargsAnykwargsAny