Dependency Packages

What Are Dependency Packages?

A dependency package is a named collection of Python dependencies (e.g., pandas>=2.0, scikit-learn) tied to a specific Python version. DAGY resolves, installs, and packages these into a ZIP file stored in S3 that can be attached to flow executions.

Creating a Package

1. Navigate to Dep Packages in the sidebar
2. Click New Package
3. Fill in:
   • Name: A descriptive name (e.g., "ML Pipeline Deps")
   • Python Version: Select 3.10, 3.11, 3.12, or 3.13
   • Description: Optional context
4. Use PyPI Search to find and add packages
5. Optionally set version constraints (e.g., >=2.0, ==1.4.0)

Dependency Versioning

Version Specification Modes

When you add a dependency, the platform provides several ways to control which version gets installed at build time:

Mode	Example Spec	Behavior
Exact Pin	==2.31.0	Installs exactly version 2.31.0. The most deterministic option.
Minor Range	>=2.31.0,<2.32.0	Allows patch updates within 2.31.x. Locks minor version to the selected minor.
Major Range	>=2.0.0,<3.0.0	Allows any 2.x.y release. Locks major version only.
Any Version	(empty)	No version constraint. The latest compatible release is resolved at build time.

Pinning with the Version Selector

Each dependency row in the UI displays cascading Major, Minor, and Build (patch) dropdowns. The lock icons to the left of Major and Minor control how broad the allowed range is:

• Unlocked Major + Unlocked Minor: Produces an exact pin (==X.Y.Z) based on your dropdown selections.
• Locked Major (only): Produces a major range (>=X.0.0,<X+1.0.0). Minor and patch dropdowns become informational.
• Locked Minor (implies locked Major): Produces a minor range (>=X.Y.0,<X.Y+1.0).

!!! tip Locking minor is the recommended default for most libraries. It allows security and bug-fix patches while preventing breaking API changes within a minor series.

The "Any Version" Option

If you do not want to pin a version at all, click the version spec badge on the right side of a dependency row to toggle it to "any." When set to "any," the version dropdowns are dimmed and the platform stores an empty version_spec for that dependency.

At build time, the package installer (uv/pip) resolves the latest version compatible with all other constraints in the package. This means the exact version installed may differ between builds if new releases are published on PyPI between build runs.

!!! warning Using "any version" trades reproducibility for convenience. If your flow relies on a specific behavior of a library, pin at least the major version to prevent unexpected breakage.

Bulk Import Versioning

When importing from a requirements.txt file, the platform parses standard pip version specifiers. All PEP 508 formats are supported:

requests==2.31.0          # exact pin
flask>=3.0,<4.0           # range
numpy                     # any version (empty spec)
scikit-learn>=1.3         # minimum version
strands-agents[a2a]       # extras supported
mylib @ https://...       # direct references

Lines starting with # (comments), -r, -c, -e, or --index-url (pip directives) are automatically ignored during parsing. Environment markers (e.g., ; python_version >= "3.8") are stripped.

Duplicate Prevention

Name Normalization (PEP 503)

Before comparing package names for duplicates, the platform normalizes names using the PEP 503 canonical form. This ensures that variations of the same package are recognized as duplicates:

• All characters are lowercased.
• Hyphens (-), underscores (_), and dots (.) are collapsed into a single hyphen.
• Extras in square brackets (e.g., [a2a]) are stripped before comparison.

Under these rules, all of the following are considered the same package:

Input	Normalized
scikit-learn	scikit-learn
scikit_learn	scikit-learn
Scikit.Learn	scikit-learn
SCIKIT_LEARN	scikit-learn

Frontend Deduplication Rules

Duplicate detection happens at the moment you add a dependency, before the package is saved:

Scenario	What Happens	User Experience
Single add of an existing package	Addition is blocked.	An info toast appears: "Package is already in the list."
Bulk import - same name, same version	Silently skipped. The existing entry is kept.	A toast shows the skip count.
Bulk import - same name, different version	A version conflict is raised.	An amber conflict banner appears (see Version Conflict Resolution).
Bulk import - intra-batch duplicate	Only the first occurrence in the batch is kept.	Subsequent duplicates are counted as skipped.

Backend Deduplication

As an additional safety net, the backend API also deduplicates dependencies on both create and update operations. The backend applies the same PEP 503 normalization and uses a last-occurrence-wins strategy. If the frontend somehow sends two entries for the same normalized name, only the last one is stored.

!!! note "Defense in Depth" Deduplication runs on both the frontend (for immediate user feedback) and the backend (as a safety net for API consumers and edge cases). This ensures consistency regardless of how dependencies are submitted.

Version Conflict Resolution

Import Conflicts

When a bulk import includes a package that already exists in the list but with a different version specifier, the platform displays an amber conflict resolution banner directly in the Dependencies section.

For each conflict, the banner shows:

• The package name.
• The existing version spec currently in the list.
• The incoming version spec from the import.
• Two action buttons: "Keep existing" and "Use incoming."

You can also resolve all conflicts at once using the "Keep All Existing" or "Use All Incoming" buttons at the top of the banner.

Build-Time Validation Conflicts

When you click Save & Build, the platform validates all dependencies against each other before starting the build. The validator checks for cross-dependency incompatibilities using the target Python version.

If the validator finds conflicts:

• The package status changes to "Invalid."
• A red validation panel appears showing specific error messages.
• The build does not start until conflicts are resolved.

Common validation errors include:

• Two packages requiring mutually exclusive versions of a shared transitive dependency.
• A package not available for the selected Python version.
• A version specifier that matches no published release on PyPI.

!!! important Validation errors do not prevent the package record from being saved. The package is created or updated first, then validation runs. If validation fails, fix the dependencies and click Save & Build again. The existing record will be updated, not duplicated.

Validation

Before building, DAGY validates that all dependencies can be resolved together without conflicts. Validation uses uv pip compile under the hood.

• Click Validate to check compatibility
• Green checkmark = all dependencies resolve cleanly
• Red X = conflicts detected; review the error messages

Validation is automatic when you click Save & Build. If dependencies changed since the last validation, it re-validates first.

Save & Build Process

The Save & Build button follows a clear three-step sequence:

Step	Action	Details
1	Save	Creates a new package record (first time) or updates the existing one (subsequent attempts). The platform tracks the package ID from the first create so retries always update.
2	Validate	Checks all dependencies for cross-compatibility against the selected Python version. If validation fails, the build is aborted and errors are displayed.
3	Build	Installs all dependencies into an isolated environment, packages them into a ZIP archive, and uploads to storage. Build progress and logs are shown in real time.

Idempotent retries: If the first Save & Build attempt fails at the validation or build step, clicking Save & Build again will update the existing package record rather than creating a duplicate. This is guaranteed by the platform tracking the package ID from the initial creation.

Build Status Progression

1. QUEUED: Build job is queued via SQS
2. INSTALLING: uv pip install runs with your dependencies
3. PACKAGING: Installed packages are zipped
4. UPLOADING: ZIP is uploaded to S3
5. COMPLETED: Package is ready for use

You can navigate away during the build. A toast notification appears when the build completes or fails.

Build Logs

Click Build Logs on a package to view the installation output in real-time. Logs are streamed from CloudWatch and auto-refresh every 3 seconds during active builds.

Using Packages with Flows

Once a package is built (status = COMPLETED), its S3 URI is available for use in flow configurations. The ZIP contains all installed Python packages and can be extracted into the runtime environment.

Best Practices

For Reproducible Environments

• Pin at least major+minor versions for all production dependencies. This balances patch-level security fixes with API stability.
• Use exact pins (==X.Y.Z) for critical libraries where even minor changes could affect behavior (e.g., ML model inference libraries).
• Avoid "any version" for production flows. Reserve it for development or experimentation.
• Rebuild after dependency changes. A saved package is not a built package. Always run Save & Build to produce a fresh archive.

For Managing Large Dependency Lists

• Use Bulk Import to add multiple dependencies at once from a requirements.txt file, rather than adding them one by one.
• Export your requirements from an existing virtual environment (pip freeze) and import the output directly.
• Review conflicts promptly. When the conflict banner appears, resolve each conflict before adding more dependencies.
• Stay under 200 dependencies per package. The platform enforces a 200-dependency limit to keep builds fast and archives manageable.

Avoiding Common Pitfalls

Pitfall	How to Avoid
Using "any" for all packages	Pin at least your top-level dependencies. Let only transitive deps float.
Duplicate names with different casings	The platform auto-normalizes names (PEP 503). No action needed.
Not rebuilding after edits	Always use Save & Build after making changes. Save alone does not produce a new archive.
Ignoring validation errors	Fix version conflicts before deploying. Invalid packages cannot be used by flow nodes.
Very broad ranges (>=0.1)	Prefer minor-locked ranges (>=X.Y.0,<X.Y+1.0) to limit unexpected upgrades.

Troubleshooting

"Dependency resolution failed"

• Check that all version constraints are compatible
• Try relaxing version pins (e.g., >=1.0 instead of ==1.0.0)
• Ensure the Python version supports all packages

"uv is not installed"

• The build environment requires uv to be available
• Verify your Lambda/container image includes uv

Build stuck in QUEUED

• Check that the SQS events queue is configured and the Lambda worker is running
• Verify DAGY_EVENTS_QUEUE_URL is set

Large package size

• Review dependencies for unnecessary packages
• Consider splitting into multiple smaller packages
• Use exact version pins to avoid pulling unnecessary transitive dependencies

Validation errors on Save & Build

• Two packages may require mutually exclusive versions of a shared transitive dependency. Try relaxing one of the conflicting version constraints.
• A package may not be available for the selected Python version. Check PyPI for version compatibility.
• A version specifier may match no published release. Verify the version number exists on PyPI.
• Validation errors do not prevent the record from being saved. Fix the issue and click Save & Build again to update the existing record.