Welcome to oryxflow documentation!
For data scientists and data engineers, oryxflow is a python library which makes it easier to build data workflows. You declare each step of your analysis as a task; oryxflow runs them in the right order, skips anything already computed, reruns exactly what a parameter, data, or code change affects, and lets you load any result by name. It also records what ran, when, and why, so “is this result stale?”, “was it produced by the current code?”, and “did I already run this?” are queries, not guesses. The payoff: no wasted recomputation, reproducible outputs you can trust, and pipelines that are easy to share — instead of a fragile chain of scripts and files you manage by hand.
Installation
Follow github instructions https://github.com/oryxintel/oryxflow#installation
Benefits of using oryxflow
See 4 Reasons Why Your Machine Learning Code is Probably Bad
Quickstart
See https://github.com/oryxintel/oryxflow/blob/master/docs/example-ml.md
Build with an AI assistant (Claude Code)
Prefer to have an AI coding assistant set up your project structure and wire the task dependencies for you? There is an official Claude Code plugin, oryxflow-claude-plugin, that does exactly that. See Using oryxflow with Claude Code.
Real-life project template
Transition to oryxflow from typical scripts
[5 Step Guide to Scalable Deep Learning Pipelines with oryxflow](https://htmlpreview.github.io/?https://github.com/d6t/d6t-python/blob/master/blogs/blog-20190813-d6tflow-pytorch.html)
Parameter Management
Intelligent parameter management is one of the most powerful features of oryxflow. New users often have questions on parameter management, this is an important section to read.
User Guide
- Quickstart
- Using oryxflow with Claude Code
- Transition to oryxflow
- Writing and Managing Tasks
- Workflow
- Running Tasks and Managing Workflows
- Logging
- Task I/O Targets
- Sharing Workflows and Outputs
- Managing Complex Workflows
- Worked example: hyperparameter tuning
- Extend the grid — nothing wasted
- What reruns, and when you do nothing
- Automatic code invalidation
- When you changed something but nothing reran
- Pinning a task: the explicit
code_version - The three exits for any code change
- Which verb, when
- Keeping old versions side by side
- Selectively reset when code versioning doesn’t apply
- The event stream: what ran, when, and why
- Where the freshness records live
- What caching does not protect against
- CLAUDE.md snippet for AI-agent projects
- Scaling up: hierarchies and independent experiments
- Advanced: Dynamic Tasks
- Advanced: Parameters
- oryxflow
- Functional Tasks
- Changelog