Aplikasi production tidak pernah diam — setiap request melewati banyak titik (load balancer, API gateway, service, database, cache) dan setiap titik menghasilkan event. Tanpa korelasi antar event, debugging masalah production terasa seperti menebak di kegelapan. Solusinya ada tiga pilar observability: structured logging (JSON, bukan plain text), metrics (agregat numerik seperti latency p99), dan traces (hierarki span antar service). Pada chapter ini kita akan mendemokan structured logger sederhana yang menghasilkan JSON dengan trace ID, menjelaskan konsep OpenTelemetry, serta memetakan stack populer untuk observability — ELK, Loki, Jaeger, dan Grafana.

Structured Logging #

Plain text log (INFO Request diterima userId=42) mudah dibaca manusia tetapi sulit diparse mesin. Structured logging mengubah setiap event menjadi JSON:

json Structured log event 
{
  "ts": "2026-01-15T08:30:00Z",
  "level": "INFO",
  "service": "user-service",
  "traceId": "trace-eaa12ac3",
  "spanId": "span-d748df24",
  "msg": "Request diterima",
  "userId": 42,
  "method": "GET",
  "path": "/users/42"
}

Setiap field dapat di-query di log aggregator: “tampilkan semua log dengan userId = 42 dan level = ERROR”. Plain text membutuhkan regex yang rentan kesalahan; JSON bersifat schema-driven.

Demo Structured Logger #

Cuplikan berikut mengimplementasikan logger sederhana yang menghasilkan JSON dengan trace ID dan span ID per request:

kotlin 
// logging_dan_observability.kt — Demo structured logging dan trace ID

import java.time.Instant
import java.util.UUID

// === Structured log event (JSON-like) ===
data class LogEvent(
    val timestamp: String,
    val level: String,
    val service: String,
    val traceId: String,
    val spanId: String,
    val message: String,
    val extra: Map<String, Any>
) {
    fun toJson(): String {
        val extraJson = extra.entries.joinToString(", ") { (k, v) ->
            val valueStr = when (v) {
                is String -> "\"$v\""
                is Number -> v.toString()
                is Boolean -> v.toString()
                else -> "\"$v\""
            }
            "\"$k\": $valueStr"
        }
        return "{\"ts\": \"$timestamp\", \"level\": \"$level\", \"service\": \"$service\", " +
            "\"traceId\": \"$traceId\", \"spanId\": \"$spanId\", \"msg\": \"$message\"" +
            (if (extra.isNotEmpty()) ", $extraJson" else "") + "}"
    }
}

// === Logger sederhana ===
class StructuredLogger(private val service: String) {
    private val traceCtx = ThreadLocal.withInitial { TraceContext("no-trace", "no-span") }

    fun setTraceContext(traceId: String, spanId: String) {
        traceCtx.set(TraceContext(traceId, spanId))
    }

    fun info(message: String, extra: Map<String, Any> = emptyMap()) =
        emit("INFO", message, extra)

    fun warn(message: String, extra: Map<String, Any> = emptyMap()) =
        emit("WARN", message, extra)

    fun error(message: String, throwable: Throwable? = null, extra: Map<String, Any> = emptyMap()) {
        val merged = extra.toMutableMap()
        if (throwable != null) merged["exception"] = throwable.javaClass.simpleName + ": " + throwable.message
        emit("ERROR", message, merged)
    }

    private fun emit(level: String, message: String, extra: Map<String, Any>) {
        val ctx = traceCtx.get()
        val event = LogEvent(
            timestamp = Instant.parse("2026-01-15T08:30:00Z").toString(),
            level = level,
            service = service,
            traceId = ctx.traceId,
            spanId = ctx.spanId,
            message = message,
            extra = extra
        )
        println(event.toJson())
    }
}

data class TraceContext(val traceId: String, val spanId: String)

// === Simulasi request dengan trace ===
class RequestHandler(private val logger: StructuredLogger) {
    fun handleGetUser(userId: Long) {
        val traceId = "trace-" + UUID.nameUUIDFromBytes(("req-" + userId).toByteArray()).toString().take(8)
        val spanId = "span-" + UUID.nameUUIDFromBytes(("get-" + userId).toByteArray()).toString().take(8)
        logger.setTraceContext(traceId, spanId)

        logger.info("Request diterima", mapOf("userId" to userId, "method" to "GET", "path" to "/users/$userId"))

        try {
            val user = fetchUser(userId)
            logger.info("User ditemukan", mapOf("userEmail" to user.email, "durationMs" to 12))
            logger.info("Response dikirim", mapOf("status" to 200, "bytesOut" to 256))
        } catch (e: RuntimeException) {
            logger.error("Gagal memproses request", e, mapOf("userId" to userId))
        }
    }

    private fun fetchUser(userId: Long): SimpleUser {
        if (userId == 99L) throw RuntimeException("User tidak ditemukan")
        return SimpleUser(userId, "[email protected]", "Budi")
    }
}

data class SimpleUser(val id: Long, val email: String, val nama: String)

fun main() {
    println("=== Logging dan Observability ===")
    println()
    println("Structured logging (JSON), trace ID, korelasi antar service.")
    println()

    // === 1. Plain log vs structured log ===
    println("=== 1. Plain vs Structured Log ===")
    println("Plain:    2026-01-15 08:30:00 INFO  Request diterima userId=42")
    println("Structured: {\"ts\":\"...\",\"level\":\"INFO\",\"msg\":\"Request diterima\",\"userId\":42}")
    println()

    // === 2. Trace ID ===
    println("=== 2. Trace ID Korelasi ===")
    val logger = StructuredLogger("user-service")
    val handler = RequestHandler(logger)
    handler.handleGetUser(42L)
    println()
    handler.handleGetUser(99L)
    println()

    // === 3. Komponen observability ===
    println("=== 3. Tiga Pilar Observability ===")
    val pillars = listOf(
        Triple("Logs", "Event diskrit", "siapa, apa, kapan (JSON)"),
        Triple("Metrics", "Aggregat numerik", "rate, latency, error per service"),
        Triple("Traces", "Span hierarki", "korelasi antar service, request flow")
    )
    pillars.forEach { (nama, deskripsi, contoh) ->
        println("  " + nama.padEnd(10) + " | " + deskripsi.padEnd(20) + " | " + contoh)
    }
    println()

    // === 4. OpenTelemetry konsep ===
    println("=== 4. OpenTelemetry Konsep ===")
    val otel = listOf(
        "Trace  = root + spans (DAG)",
        "Span   = unit kerja (request, query DB, internal call)",
        "Context = W3C Trace Context header: traceparent",
        "Baggage = cross-service metadata (user.id, locale)",
        "Exporter = OTLP (HTTP/gRPC) ke Collector"
    )
    otel.forEach { println("  " + it) }
    println()

    // === 5. Format log populer ===
    println("=== 5. Format Log Populer ===")
    val formats = listOf(
        Pair("JSON (Elastic Stack)", "Logstash, ElasticSearch, Kibana (ELK)"),
        Pair("JSON (Loki + Grafana)", "Hemat resource, query LogQL"),
        Pair("Plain (log file)", "grep-friendly, tidak perlu parser"),
        Pair("logfmt (key=value)", "Hemat byte, mirip JSON tapi ringkas")
    )
    formats.forEach { (fmt, stack) ->
        println("  " + fmt.padEnd(28) + " | " + stack)
    }
    println()

    // === 6. Jebakan umum ===
    println("=== 6. Jebakan Umum ===")
    val gotchas = listOf(
        "Log password / token -> mask sebelum log",
        "Log level INFO untuk request path dengan data sensitif -> DEBUG",
        "Tidak ada trace ID -> tidak bisa korelasi antar service",
        "Sinkronisasi log I/O blocking thread -> pakai async appender",
        "Tidak ada sampling -> volume log membengkak di produksi"
    )
    gotchas.forEach { println("  " + it) }
    println()

    // === 7. Aturan praktis ===
    println("=== 7. Aturan Praktis ===")
    val rules = listOf(
        "1. Structured logging (JSON) untuk semua log produksi",
        "2. Trace ID wajib di setiap log event untuk korelasi",
        "3. Pakai OpenTelemetry SDK (vendor-neutral)",
        "4. Sampling: 100% error, 1% success (high traffic)",
        "5. Async appender untuk hindari blocking",
        "6. Log rotation dan retention policy (30-90 hari)",
        "7. Alert berdasarkan log: ERROR spike, latency p99"
    )
    rules.forEach { println("  " + it) }
}

Output yang Dihasilkan:

text 
=== Logging dan Observability ===

Structured logging (JSON), trace ID, korelasi antar service.

=== 1. Plain vs Structured Log ===
Plain:    2026-01-15 08:30:00 INFO  Request diterima userId=42
Structured: {"ts":"...","level":"INFO","msg":"Request diterima","userId":42}

=== 2. Trace ID Korelasi ===
{"ts": "2026-01-15T08:30:00Z", "level": "INFO", "service": "user-service", "traceId": "trace-eaa12ac3", "spanId": "span-d748df24", "msg": "Request diterima", "userId": 42, "method": "GET", "path": "/users/42"}
{"ts": "2026-01-15T08:30:00Z", "level": "INFO", "service": "user-service", "traceId": "trace-eaa12ac3", "spanId": "span-d748df24", "msg": "User ditemukan", "userEmail": "[email protected]", "durationMs": 12}
{"ts": "2026-01-15T08:30:00Z", "level": "INFO", "service": "user-service", "traceId": "trace-eaa12ac3", "spanId": "span-d748df24", "msg": "Response dikirim", "status": 200, "bytesOut": 256}

{"ts": "2026-01-15T08:30:00Z", "level": "INFO", "service": "user-service", "traceId": "trace-ec23dff4", "spanId": "span-6e2d58d3", "msg": "Request diterima", "userId": 99, "method": "GET", "path": "/users/99"}
{"ts": "2026-01-15T08:30:00Z", "level": "ERROR", "service": "user-service", "traceId": "trace-ec23dff4", "spanId": "span-6e2d58d3", "msg": "Gagal memproses request", "userId": 99, "exception": "RuntimeException: User tidak ditemukan"}

=== 3. Tiga Pilar Observability ===
  Logs       | Event diskrit        | siapa, apa, kapan (JSON)
  Metrics    | Aggregat numerik     | rate, latency, error per service
  Traces     | Span hierarki        | korelasi antar service, request flow

=== 4. OpenTelemetry Konsep ===
  Trace  = root + spans (DAG)
  Span   = unit kerja (request, query DB, internal call)
  Context = W3C Trace Context header: traceparent
  Baggage = cross-service metadata (user.id, locale)
  Exporter = OTLP (HTTP/gRPC) ke Collector

=== 5. Format Log Populer ===
  JSON (Elastic Stack)         | Logstash, ElasticSearch, Kibana (ELK)
  JSON (Loki + Grafana)        | Hemat resource, query LogQL
  Plain (log file)             | grep-friendly, tidak perlu parser
  logfmt (key=value)           | Hemat byte, mirip JSON tapi ringkas

=== 6. Jebakan Umum ===
  Log password / token -> mask sebelum log
  Log level INFO untuk request path dengan data sensitif -> DEBUG
  Tidak ada trace ID -> tidak bisa korelasi antar service
  Sinkronisasi log I/O blocking thread -> pakai async appender
  Tidak ada sampling -> volume log membengkak di produksi

=== 7. Aturan Praktis ===
  1. Structured logging (JSON) untuk semua log produksi
  2. Trace ID wajib di setiap log event untuk korelasi
  3. Pakai OpenTelemetry SDK (vendor-neutral)
  4. Sampling: 100% error, 1% success (high traffic)
  5. Async appender untuk hindari blocking
  6. Log rotation dan retention policy (30-90 hari)
  7. Alert berdasarkan log: ERROR spike, latency p99

Mari kita bedah output di atas:

  1. Trace ID Korelasi: Setiap request mendapat trace ID unik (trace-eaa12ac3 untuk userId=42, trace-ec23dff4 untuk userId=99). Trace ID ini dibawa di seluruh log event — dari “Request diterima”, “User ditemukan”, hingga “Response dikirim”. Saat ada error di request userId=99, log ERROR punya trace ID yang sama, sehingga kita bisa trace seluruh lifecycle request tersebut.
  2. Field extra: setiap log event membawa field kontekstual — userId, method, path, durationMs, status, bytesOut. Field ini membuat query di log aggregator mudah: “tampilkan semua log dengan status = 200 dan durationMs > 100”.
  3. Error dengan exception: log ERROR mencatat exception: "RuntimeException: User tidak ditemukan". Tanpa trace ID, kita tidak tahu request mana yang gagal; dengan trace ID, kita bisa rekonstruksi seluruh flow.
  4. Tiga Pilar: Logs (event diskrit), Metrics (agregat numerik), Traces (hierarki span). Ketiganya saling melengkapi — logs untuk detail, metrics untuk tren, traces untuk korelasi.
  5. OpenTelemetry: vendor-neutral SDK yang menggabungkan traces, metrics, dan logs. Exporter OTLP mengirim data ke Collector, yang mengarahkan ke backend (Jaeger, Prometheus, ElasticSearch).

Trace ID dan Span ID #

Trace adalah DAG (Directed Acyclic Graph) span. Setiap span merepresentasikan unit kerja: HTTP request, query database, atau internal call. Trace ID mengidentifikasi satu request end-to-end; span ID mengidentifikasi unit kerja dalam trace.

text Trace hierarchy 
Trace abc123 (GET /api/users/42)
├── Span 1: API gateway (12 ms)
│   ├── Span 1.1: auth check (2 ms)
│   └── Span 1.2: user-service call (8 ms)
│       ├── Span 1.2.1: DB query (5 ms)
│       └── Span 1.2.2: cache lookup (1 ms)
└── Span 2: response serialization (1 ms)

W3C Trace Context mengoper trace via HTTP header traceparent: 00-{traceId}-{spanId}-01. Setiap service yang menerima request melanjutkan trace dengan span baru.

OpenTelemetry SDK #

OpenTelemetry (OTel) adalah proyek CNCF yang menyatukan traces, metrics, dan logs dalam satu SDK vendor-neutral. Library Kotlin populer: io.opentelemetry:opentelemetry-sdk. Konfigurasi:

kotlin OpenTelemetry setup 
val otel = OpenTelemetrySdk.builder()
    .setTracerProvider(
        SdkTracerProvider.builder()
            .addSpanProcessor(BatchSpanProcessor.builder(otlpExporter).build())
            .build()
    )
    .build()

val tracer = otel.tracerProvider.get("my-app")
val span = tracer.spanBuilder("handleGetUser").startSpan()
try {
    // business logic
} finally {
    span.end()
}

Stack Observability Populer #

StackLogsMetricsTracesCocok untuk
ELK (Elasticsearch)✓✓ (terbatas)✓ (APM)All-in-one
Loki + Grafana + Tempo✓✓ (Prometheus)✓Cloud-native, hemat resource
Datadog✓✓✓SaaS, biaya tinggi
OpenTelemetry + Jaeger + Prometheus✓✓✓Vendor-neutral, self-hosted

Referensi Lanjutan #

Selanjutnya #

Logging terstruktur memberikan visibilitas event, tetapi load balancer dan orchestrator butuh sinyal cepat untuk menentukan apakah service siap menerima traffic. Bab berikutnya membahas health checks — topik pada chapter Health Checks.