EXPERIENCE / RUIZHANG ZHOU
Applied AI / AI Platform Engineer
KI:connect AI Platform
I contribute to a multi-tenant platform that gives users and applications access to LLMs through a web interface and APIs. My work spans provider behavior, request and response handling, metering, integration reviews, and service diagnostics.
- Worked on OpenAI-compatible Chat Completions and Responses APIs, including raw request forwarding and streamed response handling.
- Added and validated provider-specific model capabilities and multimodal token estimation where provider behavior differs.
- Worked on usage and cost calculation across model deployments, including cached input tokens and cache-write pricing.
- Contributed to quota handling and administrative controls for model access and deployment settings.
- Reviewed web-search tool integration, provider boundaries, and tenant eligibility for agent workflows; distinguished the application contract from provider-specific configuration.
- Worked through RAG integration and rollout requirements, including document ingestion, queue and storage dependencies, service authentication, and container runtime checks.
- Traced release and request failures through CI/CD, Nomad, IIS, reverse proxies, application code, and providers; separated build, deployment, health-check, and external-access evidence.
- Used Grafana dashboards and OpenSearch logs to investigate request behavior and load-test failures; correlated telemetry across service boundaries.
- Investigated Shibboleth/SAML login issues using service-provider metadata and time-correlated logs. Contributed to deployment configuration and the Vue-based admin interface.
TalkEcho
A desktop meeting-caption and dictation application using configurable speech-to-text and LLM providers. My contributions focus on making live workflows more reliable and usable.
- Added dictation and Windows tray workflows, with hotkey and window-behavior fixes.
- Improved long-meeting request handling with a coalescing backlog, timeouts, and retries for transient failures.
- Moved provider credentials to the operating system's credential store.
GitHub Repository (fork of Pluely)
LLM Gateway
A task-aware model router for personal automation jobs. Applications request a task type, while routing policy chooses from the available models.
- Refreshes its model catalog from a live models endpoint and retains the previous configuration when discovery fails.
- Falls back on rate limits, timeouts, capacity errors, and unavailable models.
- Limits streaming failover to the period before the first output chunk, avoiding mixed responses from two models.
Additional API testing experiment: CompatBench.
Time Series Generative Models for Traffic Scenario Generation
Implement and train time series generative models to synthesize realistic vehicle trajectory data for autonomous vehicle testing. Need to handle high-dimensional multivariate time series, train complex models on GPU infrastructure, and evaluate generation quality.
- Model Implementation: Implemented TimeGAN (GAN-based) and Diffusion-TS (diffusion-based) architectures in PyTorch for multivariate time series generation
- Data Pipeline: Processed IKA real-world driving datasets (inD, rounD, exiD) with sliding window extraction, normalization, and data augmentation
- GPU Training: Trained models on NVIDIA H100 and Quadro RTX 6000 GPUs with hyperparameter optimization, learning rate scheduling, and early stopping
- Model Evaluation: Implemented evaluation pipeline using PCA, t-SNE visualization, and statistical metrics to assess trajectory realism and diversity
- Integration: Built conversion pipeline to XML format for CPM Remote platform, enabling generated scenarios to be used in autonomous vehicle simulation
The thesis connects ML model development with the data processing and evaluation needed to study generated trajectories.
LLM & GNN Research for Medical Image Analysis
Explore chest X-ray report generation by combining vision and language models in a research setting.
- Developed a pipeline combining BLIP (vision-language model) with LLMs for chest X-ray report generation
- LLM Deployment & Serving: Deployed LLaMA models on GPU cluster using Ollama, built REST API service with FastAPI for internal research access, managed model inference and user requests
- Conducted research on integrating Knowledge Graphs with GNNs to improve medical domain understanding
- Ran large-scale experiments on GPU cluster infrastructure for model training and evaluation, experimented with vLLM for high-throughput inference
- Built end-to-end LLM chat platform: GPU deployment → API service → web interface for research demonstrations
This research experience introduced me to model serving, multimodal pipelines, and GPU-based experimentation.