API Overview

Gigantics provides REST API endpoints that allow programmatic access to datasets and pipelines without requiring UI authentication. This enables integration with external systems, automated workflows, and data consumption by applications.

Available Endpoints

Dataset Endpoints

Download dataset data in various formats using GET requests:

GET /api/{org}/{proj}/model/{model}/dataset/{id}

Features:

• Multiple output formats (SQL, JSON, CSV, Oracle Loader)
• Entity filtering
• Metadata access
• Record limiting

See API Endpoint Assignment for details.

Pipeline Endpoints

Trigger pipeline execution using POST requests:

POST /api/{org}/{proj}/model/{model}/pipeline/{id}

Features:

• Asynchronous job execution
• Job status tracking via returned job URL
• Integration with external orchestrators

See API Endpoint Assignment for details.

Job Status Endpoint

Check the status of jobs (publicly accessible, no authentication required):

GET /api/job/{jobId}

Returns:

• Job status (running, completed, failed)
• Progress percentage for running jobs
• Failure reasons for failed jobs
• Result data for completed jobs

Authentication

All dataset and pipeline endpoints require API key authentication. Two methods are supported:

1. Query parameter: ?api_key={your-key}
2. Authorization header: Authorization: Bearer {your-key}

API keys are created and managed through the UI. See Authentication for details.

Getting Started

1. Create an API key: Navigate to Project ? API Keys and create a key
2. Assign to a resource: Click the share icon on a dataset or pipeline
3. Get the endpoint URL: The URI is displayed in the UI
4. Make requests: Use the API key with the endpoint URL

Key Concepts

Endpoints vs Resources

• Resources (datasets, pipelines) are the actual data/workflows
• Endpoints are the API URLs that expose resources
• One resource can have one endpoint
• Multiple API keys can be assigned to one endpoint

API Keys

• Format: {prefix}-{key} (31 characters total)
• Shown only once during creation
• Can be active or inactive
• Stored as bcrypt hashes

Multiple Keys per Endpoint

You can assign multiple API keys to a single endpoint, enabling:

• Zero-downtime key rotation
• Multi-party access control
• Backup keys

Any assigned key will authenticate the endpoint. See Multiple API Keys per Endpoint for details.

Documentation

• API Endpoint Assignment - Create and manage endpoints
• Multiple API Keys per Endpoint - Assign multiple keys
• Authentication - Understand API key authentication
• Access Control - Permissions and security
• Error Handling - Handle API errors
• Rate Limiting - Current limitations

Examples

Download Dataset as SQL

curl "https://yourserver/api/org/proj/model/1/dataset/42?api_key=abc123xyz-def456uvw-ghi789rst&format=sql"

Download Dataset as JSON Zip

curl "https://yourserver/api/org/proj/model/1/dataset/42?api_key=abc123xyz-def456uvw-ghi789rst&format=json-zip" \
  -o dataset.zip

Trigger Pipeline

curl -X POST "https://yourserver/api/org/proj/model/1/pipeline/5?api_key=abc123xyz-def456uvw-ghi789rst"

Check Job Status

curl "https://yourserver/api/job/507f1f77bcf86cd799439011"

Best Practices

1. Secure your keys - Never commit API keys to version control
2. Use HTTPS - Always use encrypted connections
3. Rotate regularly - Create new keys and phase out old ones
4. Monitor usage - Check call counts and last-used times
5. Use descriptive purposes - Name keys by their use case
6. Handle errors - Implement retry logic for transient failures