Dagychu buyer guide
The hidden execution platform your team already maintains
A background job rarely stays just a job. As the business begins to depend on it, teams add status tables, retry endpoints, log searches, recovery scripts and an operator screen. The runner was only the beginning.
What this page covers
This is a diagnostic guide, not an argument to replace every runner. Keep cron, Celery, Kubernetes Jobs, Airflow or another system where it already answers both the execution and operational questions your team has.
Evaluate a separate control plane when a real workflow can start, but understanding its state or recovering it still depends on the original author, manual SQL or a collection of internal tools.
How one job becomes an internal platform
The platform usually appears incrementally. Every addition is reasonable on its own; together they become infrastructure with its own state model, permissions and operating procedures.
| Stage | What the team adds | The next unanswered question |
|---|---|---|
| Script or cron entry | A process that starts work | Did the business operation finish and produce the expected result? |
| Queue or job runner | Dispatch, workers and process retries | Which business request does this worker attempt belong to? |
| Status table | Application-specific states and timestamps | Which step failed, with which input and after which side effects? |
| Retry endpoint | A way to submit the work again | Is repeating the whole operation safe, or should only one job run again? |
| Admin screen | Search, status changes and recovery buttons | Who may act, what changed and how will another team operate it? |
| On-call runbook | Manual SQL, log queries and escalation steps | Why does recovery still require the person who built the workflow? |
Signs that you already maintain this platform
The relevant signal is not the number of jobs. It is the amount of operational knowledge and custom machinery required when one of them fails.
- A product request outlives the HTTP request that created it, and users need a durable status or result.
- Support or operations must ask an engineer whether a task is still running, stuck or safe to repeat.
- Each new import, export, reconciliation or AI workflow creates another status table or set of endpoints.
- Inputs, job status, errors and logs live in different systems and must be joined mentally during an incident.
- The team can restart a process but cannot immediately explain which completed effects would be repeated.
- Recovery regularly uses manual SQL, shell access, an internal script or knowledge held by one person.
- A simple admin page has grown into search, permissions, history and a recovery handbook.
What the hidden platform costs, even when its license is zero
The cost is not a generic industry benchmark and should not be reduced to an invented dollar figure. Measure the engineering and operating work your own workflow creates.
| Component | Why it keeps growing | Evidence to measure |
|---|---|---|
| Status tables | Every workflow reopens questions about states, timeouts and permitted transitions. | Schema and migration work, state-repair incidents, service-specific conventions. |
| Retry APIs | A retry without the original input and effect boundary may be a new attempt based on a guess. | Custom endpoints, authorization, duplicate-effect incidents and operator instructions. |
| Operator admin UI | A status page becomes search, permissions, history, recovery controls and documentation. | Frontend/backend maintenance and the number of workflows with separate screens. |
| Manual recovery | An engineer reconstructs inputs, completed steps and side effects before deciding what may run again. | Time to explain the state, people involved and whether someone other than the author can recover it. |
| Scattered observability | Infrastructure logs show processes; the business asks about a particular customer operation and its result. | Tools and queries opened during an incident, plus missing links between them. |
What Dagychu takes—and what remains your responsibility
Dagychu provides an execution-control boundary around Python and Bash work. It does not infer the business meaning of external effects or turn unsafe code into an exactly-once operation.
| Dagychu owns | Your job or application still owns |
|---|---|
| Task and job-run identity across execution state, inputs, outputs, logs and operator views. | Business identity, acceptance criteria and the meaning of a successful result. |
| YAML-defined pipelines and local or Docker execution of Python and Bash jobs. | Job code, dependencies, credentials and domain behavior. |
| Persisted input, output_json, attempts, errors and execution history. | Which fields may contain sensitive data and how long business data should be retained. |
| Fixed or exponential retries, per-job rerun and reconciliation after infrastructure failure. | Idempotency, deduplication or compensation for writes to external systems. |
| Cron, manual, webhook, dependency, SLA and deadline plans. | The business deadline and what action is required when it is missed. |
| Investigation graph, operations timeline and the ability to freeze external triggers. | The decision that a side effect is safe to repeat and verification of the final business result. |
Hidden platform, runner and Dagychu
A runner answers whether a process can execute. An operating model must also answer which business operation it belongs to, what happened and what may safely happen next.
| Need | Typical hidden platform | Runner alone | Dagychu |
|---|---|---|---|
| Identity of one business operation | Ad-hoc identifiers across application tables | Queue, process or pod identifier | Task and job-run records connected to execution evidence |
| Input of the failed step | Captured by service-specific code | Depends on the integration and logging | Persisted job input in the control plane |
| Targeted recovery | Custom endpoint or script | Resubmit a message or create a new job | Inspect and rerun selected job work |
| Operator investigation | Custom UI and runbook | External logs and infrastructure dashboards | Graph, timeline, inputs, outputs and logs in one operating surface |
| Backend execution contract | Each service implements create/status/result | Not normally the runner's responsibility | Enterprise External Execution API with status, result, callbacks and rate limits |
| Shared organizational access | Home-grown admin permissions | Infrastructure permissions are not necessarily business permissions | Enterprise advanced RBAC and OIDC SSO |
| Reports from output_json | Custom SQL or application reporting | Separate reporting integration | Enterprise saved Table Reports and Job Output View Designer |
| Ownership and lock-in | You own every internal convention | You own the runner integration | Dagychu application is closed; your Python/Bash jobs and pipeline definitions remain yours |
Do not add Dagychu when
A separate control plane is additional infrastructure. It should remove more operational machinery than it introduces.
- Failures are cheap, jobs are independent and a useful alert plus a simple rerun is sufficient.
- Your current system already lets an operator find the business request, original input, logs, outcome and safe recovery action.
- The primary problem is warehouse interval scheduling, provider integrations and historical backfill; evaluate Airflow for that workload.
- The primary problem is long-lived application sagas designed in code; evaluate systems built for that programming model.
- You require an open-source application tree that your organization can fork and maintain independently.
- The proposed project starts by replacing every runner instead of isolating one workflow boundary.
Evaluate one real workflow, not a category
Do not compare an empty Dagychu installation with a mature production runner. Select work your team already has to investigate or recover by hand.
- Choose one recurring Python or Bash workflow and write down its current failure and recovery procedure.
- Name its business identity, inputs, expected output, credentials and external side effects.
- Model only that workflow as a Dagychu pipeline; leave unrelated jobs in their current systems.
- In an isolated environment, run one successful execution and verify its business result.
- Introduce a controlled failure after an intermediate job and ask someone other than the implementer to explain the state.
- Recover using the agreed idempotency or reconciliation rules, then verify the final result for missing or duplicate effects.
- Compare investigation effort, recovery correctness and the custom status, retry and admin code that would remain.
Frequently asked questions
Is Dagychu a replacement for Celery or Kubernetes Jobs?
Not as a drop-in adapter. Dagychu operates a separately configured workflow through its own local or Docker execution model; it does not import the history of an existing Celery worker or Kubernetes Job.
Does a targeted rerun guarantee that side effects happen exactly once?
No. Dagychu controls which execution work is retried, but the job must implement idempotency, deduplication, compensation or another business-safe recovery rule.
When is Community enough?
Use Community to evaluate and operate the core execution lifecycle for a team: pipelines, schedules, workers, logs, investigation and recovery. Enterprise is relevant when Dagychu becomes shared organizational infrastructure requiring advanced access, external APIs, reporting or cross-system monitoring.
Should we migrate every background job?
No. Start with one workflow that has a costly investigation or recovery path. Keep existing tooling wherever it already provides the operating model you need.
Evaluate Dagychu
Start with one real Python or Bash workflow. Verify a successful run, inspect a controlled failure, and test the recovery boundary against your own business result.