Skip to main content

​
Infrastructure Agent API

The Infrastructure Agent is a Go-based service that collects health metrics from Kubernetes infrastructure components and exposes them via HTTP endpoints. Base URL: http://logclaw-agent:8080 Dashboard proxy: /api/agent/*

​
Health Check

GET /health
Liveness probe endpoint.

​
Response

{
  "status": "ok"
}

​
Readiness Check

GET /ready
Readiness probe endpoint. Returns 200 only when all collectors are initialized.

​
Response (Ready)

{
  "status": "ready",
  "collectors": {
    "kafka": "initialized",
    "flink": "initialized",
    "opensearch": "initialized",
    "eso": "initialized"
  }
}

​
Response (Not Ready)

{
  "status": "not_ready",
  "collectors": {
    "kafka": "initialized",
    "flink": "initializing",
    "opensearch": "initialized",
    "eso": "error"
  }
}

​
Infrastructure Metrics

GET /metrics
Returns aggregated infrastructure metrics from all collectors. Used by the Dashboard’s pipeline monitoring view.

​
Response

{
  "kafka": {
    "status": "healthy",
    "brokers": 3,
    "topics": {
      "raw-logs": {
        "partitions": 6,
        "consumerLag": 150
      },
      "enriched-logs": {
        "partitions": 6,
        "consumerLag": 0
      }
    }
  },
  "flink": {
    "status": "healthy",
    "jobs": [
      {
        "name": "logclaw-anomaly-detection",
        "state": "RUNNING",
        "taskManagers": 2,
        "uptime": "48h30m"
      }
    ]
  },
  "opensearch": {
    "status": "green",
    "nodes": 6,
    "indices": 15,
    "totalDocs": 1542000,
    "totalSizeBytes": 2147483648,
    "activePrimaryShards": 15,
    "activeShards": 30
  },
  "eso": {
    "status": "healthy",
    "externalSecrets": [
      {
        "name": "logclaw-ticketing-credentials",
        "status": "SecretSynced",
        "lastSyncTime": "2024-03-01T15:00:00Z"
      }
    ]
  }
}

​
Metric Fields

​
Kafka

FieldTypeDescription
statusstringOverall Kafka health: healthy, degraded, down
brokersintegerNumber of active Kafka brokers
topics[].partitionsintegerNumber of partitions per topic
topics[].consumerLagintegerConsumer lag (unconsumed messages)
FieldTypeDescription
statusstringFlink health: healthy, degraded, down
jobs[].namestringJob deployment name
jobs[].statestringJob state: RUNNING, FAILED, SUSPENDED
jobs[].taskManagersintegerNumber of active task managers

​
OpenSearch

FieldTypeDescription
statusstringCluster health: green, yellow, red
nodesintegerTotal cluster nodes
totalDocsintegerTotal indexed documents
totalSizeBytesintegerTotal index storage in bytes

​
External Secrets Operator

FieldTypeDescription
statusstringESO health: healthy, degraded, down
externalSecrets[].statusstringSync status: SecretSynced, SecretSyncError
externalSecrets[].lastSyncTimestringISO-8601 timestamp of last sync

​
Configuration

​
Environment Variables

VariableRequiredDefaultDescription
LOGCLAW_TENANT_IDYes—Tenant identifier for filtering CRDs
LOGCLAW_NAMESPACENoCurrent namespaceKubernetes namespace to watch
PORTNo8080HTTP server port