LFS Client SDKs

KafScale provides LFS client SDKs in four languages. Each SDK handles envelope encoding/decoding, HTTP upload to the LFS proxy, and S3 object resolution with checksum validation.

Go (built-in)

The Go SDK lives in pkg/lfs/ and is used internally by the LFS proxy, console, and processors.

import "github.com/KafScale/platform/pkg/lfs"

// Produce a large file through the LFS proxy HTTP API.
producer := lfs.NewProducer("http://localhost:8080")
result, err := producer.Produce(ctx, "demo-topic", "large-file.bin", fileReader)

// Resolve a consumed envelope directly from S3 with local checksum validation.
s3Client, err := lfs.NewS3Client(ctx, lfs.S3Config{
    Bucket:         "my-bucket",
    Region:         "us-east-1",
    Endpoint:       "http://localhost:9000", // optional, for MinIO
    ForcePathStyle: true,                    // optional, for MinIO
})
consumer := lfs.NewConsumer(s3Client)
envelopeBytes, _ := lfs.EncodeEnvelope(result.Envelope)
_, payload, err := consumer.Unwrap(ctx, envelopeBytes)

Trust model: Kafka is authoritative; S3 is untrusted storage. Current SDK resolvers fetch directly from S3 and validate the envelope checksum locally. HTTP clients that use POST /lfs/download must pass integrity.sha256 AND integrity.size from the envelope so the proxy can verify before streaming. See the LFS Proxy doc for the full design.

Java

Maven-based SDK with retry/backoff and configurable HTTP timeouts.

<dependency>
    <groupId>org.kafscale</groupId>
    <artifactId>lfs-sdk</artifactId>
    <version>0.1.0</version>
</dependency>

import org.kafscale.lfs.LfsProducer;

LfsProducer producer = new LfsProducer.Builder()
    .proxyUrl("http://localhost:8080")
    .topic("demo-topic")
    .retryAttempts(3)
    .connectTimeoutMs(5000)
    .build();

producer.upload("video.mp4", inputStream, "video/mp4");

Build from source:

cd lfs-client-sdk/java
mvn clean package

Python

Pip-installable SDK with retry/backoff and envelope codec.

pip install lfs-client-sdk/python/

from lfs_sdk import LfsProducer

producer = LfsProducer(
    proxy_url="http://localhost:8080",
    topic="demo-topic",
    max_retries=3,
)
producer.upload("scan.dcm", open("scan.dcm", "rb"), content_type="application/dicom")

JavaScript (Node.js)

TypeScript SDK with streaming upload support.

cd lfs-client-sdk/js
npm install && npm run build

import { LfsProducer } from '@kafscale/lfs-sdk';

const producer = new LfsProducer({
  proxyUrl: 'http://localhost:8080',
  topic: 'demo-topic',
});
await producer.upload('payload.bin', readStream);

Browser SDK

Lightweight SDK for single-page applications. Uploads files directly from the browser to the LFS proxy HTTP API.

import { LfsProducer } from '@kafscale/lfs-browser-sdk';

const producer = new LfsProducer({
  proxyUrl: 'http://lfs.example.com',
  topic: 'uploads',
});

// Upload from a file input
const file = document.getElementById('fileInput').files[0];
await producer.upload(file.name, file);

See the E72 browser demo (examples/E72_browser-lfs-sdk-demo/) for a complete single-page application.

Demo applications

Run any demo:

make lfs-demo          # Core LFS stack
make lfs-demo-medical  # E60 medical demo
make lfs-demo-video    # E61 video demo

Related docs

• LFS Demos: runnable demos for each SDK and industry scenario
• LFS Proxy: proxy architecture and configuration
• LFS Helm deployment: Kubernetes deployment guide