Running AMTD Fetcher as a Kubernetes CronJob with Devtron#

This guide demonstrates how to deploy an AMTD (Advanced Moving Target Defense) data fetcher as a scheduled CronJob in Kubernetes using Devtron’s GitOps-based deployment platform. The solution fetches security data from Morphisec’s API every 15 minutes and streams it to NATS for real-time processing.

Architecture Overview#

The AMTD fetcher solution consists of:

Python Script: fetch-amtd-data.py that polls the Morphisec API
NATS Integration: Streams data to NATS subjects for real-time processing
Kubernetes CronJob: Runs the fetcher every 15 minutes
Persistent Storage: Stores NDJSON files and health check data
Secret Management: Secure storage of NATS credentials

Prerequisites#

Kubernetes cluster with Devtron installed
NATS account with valid credentials (JWT + NKEY)
Morphisec API credentials (Client ID and Secret Key)
Docker registry for container images
Git repository connected to Devtron

Step 1: Prepare Your Kubernetes Manifests#

1.1 Create Secret for NATS Credentials#

Store your NATS credentials securely in a Kubernetes Secret:

1
apiVersion: v1
2
kind: Secret
3
metadata:
4
  name: amtd-nats-creds
5
type: Opaque
6
stringData:
7
  nats.creds: |-
8
    -----BEGIN NATS USER JWT-----
9
    eyJ0eXAiOiJKV1QiLCJhbGciOiJlZDI1NTE5LW5rZXkifQ...
10
    -----END NATS USER JWT-----
11
    -----BEGIN USER NKEY SEED-----
12
    SUACSSL3UAHUDXKFSNVUZRF5UHPMSGZ6H...
13
    -----END USER NKEY SEED-----

This secret will be mounted at /app/nats-credentials/nats.creds in your container.

1.2 Create PersistentVolumeClaim for Data Storage#

Define a PVC to store NDJSON output and health check files:

1
apiVersion: v1
2
kind: PersistentVolumeClaim
3
metadata:
4
  name: amtd-data-pvc
5
spec:
6
  accessModes: [ReadWriteOnce]
7
  resources:
8
    requests:
9
      storage: 1Gi

1.3 Define the CronJob#

Create the main CronJob manifest that schedules your fetcher:

1
apiVersion: batch/v1
2
kind: CronJob
3
metadata:
4
  name: amtd-fetcher
5
spec:
6
  schedule: "*/15 * * * *" # Run every 15 minutes
7
  concurrencyPolicy: Allow # Allow overlapping runs
8
  successfulJobsHistoryLimit: 3 # Keep last 3 successful runs
9
  failedJobsHistoryLimit: 1 # Keep last failed run
10
  startingDeadlineSeconds: 100 # Skip if missed by >100s
11
  suspend: false # Ensure it's active
12
  jobTemplate:
13
    spec:
14
      template:
15
        spec:
16
          restartPolicy: OnFailure # Retry on container crash
17
          containers:
18
            - name: fetcher
19
              image: your-registry/amtd-fetcher:latest
20
              command: ["python3", "fetch-amtd-data.py"]
21
              env:
22
                # Morphisec API Configuration
23
                - name: API_URL
24
                  value: "https://ap-1.morphisec.cloud/api/v1/authenticate"
25
                - name: BASE_URL
26
                  value: "https://ap-1.morphisec.cloud/api/v1"
27
                - name: CLIENT_ID
28
                  value: "7c68f48e-d10c-4111-818f-8f20513b02a3"
29
                - name: SECRET_KEY
30
                  value: "9b230bf1-2926-4972-887c-c05fdc49599f"
31

32
                # Data Output Configuration
33
                - name: ENABLE_NATS
34
                  value: "true"
35
                - name: ENABLE_OPENSEARCH
36
                  value: "false"
37
                - name: ENABLE_FILE_STORAGE
38
                  value: "false"
39

40
                # NATS Configuration
41
                - name: NATS_SERVERS
42
                  value: "tls://connect.ngs.global:4222"
43
                - name: NATS_SUBJECT_PREFIX
44
                  value: "amtd"
45
                - name: NATS_CREDS_PATH
46
                  value: "/app/nats-credentials/nats.creds"
47

48
                # OpenSearch Configuration (disabled)
49
                - name: OPENSEARCH_HOST
50
                  value: ""
51
                - name: OPENSEARCH_PORT
52
                  value: "9200"
53
                - name: OPENSEARCH_USER
54
                  value: ""
55
                - name: OPENSEARCH_PASSWORD
56
                  value: ""
57

58
                # AWS Configuration (for future use)
59
                - name: AWS_REGION
60
                  value: ""
61
                - name: AWS_ACCESS_KEY_ID
62
                  value: ""
63
                - name: AWS_SECRET_ACCESS_KEY
64
                  value: ""
65

66
                # Timing Configuration
67
                - name: REQUEST_TIMEOUT
68
                  value: "3600" # 1 hour timeout
69
                - name: REFRESH_INTERVAL
70
                  value: "900" # 15 minute refresh
71

72
              volumeMounts:
73
                - name: nats-creds
74
                  mountPath: /app/nats-credentials/nats.creds
75
                  subPath: nats.creds
76
                  readOnly: true
77
                - name: data-dir
78
                  mountPath: /app/data
79

80
          volumes:
81
            - name: nats-creds
82
              secret:
83
                secretName: amtd-nats-creds
84
                items:
85
                  - key: nats.creds
86
                    path: nats.creds
87
            - name: data-dir
88
              persistentVolumeClaim:
89
                claimName: amtd-data-pvc

Step 2: Configure in Devtron#

2.1 Push Manifests to Git#

Combine all three manifests into a single YAML file and commit to your Git repository:

1
# Create the manifest file
2
cat > k8s/cronjob-amtd-fetcher.yaml << 'EOF'
3
# Secret, PVC, and CronJob manifests here...
4
EOF
5

6
# Commit and push
7
git add k8s/cronjob-amtd-fetcher.yaml
8
git commit -m "Add AMTD fetcher CronJob manifests"
9
git push origin main

2.2 Configure Devtron Application#

Navigate to Devtron UI:
- Go to Applications → Create New Application
- Select your Git repository
- Choose the appropriate environment
Select Base Deployment Template:
- Go to App Configuration → Base Deployment Template
- Select “Job and Cronjob” template
- You’ll see cronjobConfigs and jobConfigs sections
Configure CronJob Settings:
- Set Kind to CronJob
- Reference your manifest under cronjobConfigs
- Configure any additional overrides

2.3 Deploy the Application#

Save your configuration
Click “Deploy” to create the resources
Navigate to Workloads → CronJobs to verify deployment

Step 3: Alternative Deployment with Helm#

If you prefer a Helm-based approach, use Devtron’s generic Helm chart:

3.1 Select Generic Helm Chart#

In Devtron’s Chart Store, select devtron-charts/devtron-generic-helm
Configure the chart values

3.2 Configure values.yaml#

1
cronjob:
2
  enabled: true
3
  spec:
4
    schedule: "*/15 * * * *"
5
    jobTemplate:
6
      spec:
7
        template:
8
          spec:
9
            containers:
10
              - name: fetcher
11
                image: your-registry/amtd-fetcher:latest
12
                # ... rest of the configuration

3.3 Deploy via Helm#

Devtron will render and apply your CronJob using the Helm chart.

Monitoring and Troubleshooting#

Check CronJob Status#

1
# View CronJob details
2
kubectl get cronjob amtd-fetcher -n your-namespace
3

4
# Check recent job executions
5
kubectl get jobs -n your-namespace | grep amtd-fetcher
6

7
# View pod logs from the last run
8
kubectl logs -n your-namespace -l job-name=amtd-fetcher-xxxxx

Common Issues and Solutions#

NATS Connection Failures:
- Verify NATS credentials are correctly formatted
- Check network connectivity to NATS server
- Ensure TLS is properly configured
API Authentication Errors:
- Verify Morphisec API credentials
- Check API endpoint URLs
- Ensure network access to Morphisec cloud
Storage Issues:
- Verify PVC is bound and accessible
- Check disk space availability
- Ensure proper write permissions

Health Monitoring#

The fetcher writes health check files to /app/data/health/:

last_run.json: Timestamp and status of last execution
metrics.json: Performance metrics and counters

Security Best Practices#

Credential Management:
- Never hardcode sensitive credentials
- Use Kubernetes Secrets for all sensitive data
- Rotate credentials regularly
Network Security:
- Use TLS for all external connections
- Implement network policies if needed
- Monitor outbound connections
Resource Limits:
- Set appropriate CPU and memory limits
- Configure resource requests for scheduling
- Monitor resource usage patterns
Access Control:
- Use RBAC for CronJob management
- Limit who can view/edit secrets
- Audit configuration changes

Advanced Configuration#

Custom Fetch Intervals#

Modify the schedule using standard cron syntax:

Every hour: "0 * * * *"
Every 30 minutes: "*/30 * * * *"
Daily at 2 AM: "0 2 * * *"

Multi-Environment Deployment#

Use Devtron’s environment overrides to deploy to multiple environments with different configurations:

1
env:
2
  - name: NATS_SUBJECT_PREFIX
3
    value: "amtd.dev"
4

5
# prod-overrides.yaml
6
env:
7
  - name: NATS_SUBJECT_PREFIX
8
    value: "amtd.prod"

Scaling Considerations#

For high-volume deployments:

Use separate CronJobs for different data types
Implement parallel processing within the fetcher
Consider using Kubernetes Jobs with parallelism
Monitor and adjust based on API rate limits

Conclusion#

Deploying the AMTD fetcher as a Kubernetes CronJob with Devtron provides a robust, GitOps-driven solution for scheduled security data collection. The combination of Kubernetes native scheduling, Devtron’s deployment management, and NATS streaming creates a scalable and maintainable architecture for security data pipelines.

Key benefits of this approach:

Automated Scheduling: Reliable execution every 15 minutes
GitOps Workflow: Version-controlled configuration
Secure Credential Management: Kubernetes Secrets integration
Scalable Architecture: Easy to extend and modify
Monitoring Integration: Native Kubernetes observability

This solution seamlessly integrates with your existing security infrastructure while leveraging modern DevOps practices for deployment and management.