AI Infrastructure Architect

NeuReality is seeking a Lead System Architect to join our system architecture team and help define NR-NEXUS, our next-generation AI inference platform.

Responsibilitie

• Lead the software architecture and technical roadmap for NeuReality’s NR-Nexus

• Write system specifications for NR-Nexus product

• Research AI infrastructure, SaaS platforms, model serving, and inference trend

• Work with engineering to translate technical capabilities into product value

• Work closely with engineering teams to optimize performance, scalability, and feature delivery

• Define performance goals and lead profiling, benchmarking, and optimization efforts for GenAI and distributed AI workloads

• Collaborate with customers, partners, and open-source communities to ensure ecosystem compatibility and adoption

• Mentor software engineers and provide technical leadership

Requirements

• 7+ years of software engineering experience, including 3+ years in software architecture or technical leadership.

• Deep understanding of Gen AI/LLM infrastructure and distributed workloads- Must

• Strong experience with Kubernetes-based platforms and cloud-native architecture.

• Experience designing management software or SaaS platforms for production systems.

• Strong background in distributed systems, microservices, APIs, and automation.

• Hands-on experience with observability stacks, monitoring, logging, alerting, and SLA/SLO tracking.

• Experience with CI/CD, deployment automation, upgrades, and rollback mechanisms.

• Good understanding of security, authentication, authorization, and integration with customer data center environments.

Nice to have

• Deep understanding of GenAI / LLM inference infrastructure, including model serving, scaling, batching, latency, throughput, and resource utilization.

• Experience with production AI inference clusters using GPUs, AI accelerators, or other specialized compute infrastructure.

• Understanding of how distributed inference systems operate, including scheduling, load balancing, autoscaling, failover, and cluster-level observability.

• Experience with LLM serving frameworks such as vLLM, Triton Inference Server, TensorRT-LLM, or similar.

• Familiarity with GPU/accelerator orchestration, device plugins, resource scheduling, and cluster capacity planning.

• Familiarity with GPU communication technologies such as GPUDirect RDMA, NCCL, NVLink, or UALink.

• Experience optimizing communication for distributed AI/ML workloads.

• Knowledge of Prometheus, Grafana, OpenTelemetry, Helm, Argo CD, Istio, KServe, Kubeflow, or similar tools.

• Experience deploying software in on-prem, edge, private cloud, or hybrid environments.