Solution Features
AI Orchestrator
AI Orchestrator coordinates deployment, integration, and interaction of AI workloads across nodes in the cluster. This includes orchestrating AI models, managing data flow, and allocating and monitoring computing resources
- Cloud Native Solution - The IKTARA AI Platform is a cloud-native solution embracing microservices architecture, containerized design, and CI/CD methodologies. The platform supports containerized AI workloads.
- Solution Components - The solution's components include the ML Orchestrator, Training Server, Inference Server, Data Pipeline Engine, Repository Management, and Structured/Columnar/Vector Data Stores.
- Model Lifecycle Management – The solution supports AI model lifecycle management for different stages such as Data Gathering & Preparation, Model Training, Model Evaluation, Model Deployment, and Model Governance. This enables seamless exploration and deployment of multiple AI models.
Distributed Machine Learning
ML algorithms are becoming complex and their training and inference tasks now need multiple GPUs. Distributed ML involves utilizing varied parallelism techniques to use distributed computing resources.
- Parallelism Approaches - Support for different parallelism approaches, such as Data Parallel, Pipelined Model Parallel, and Tensor Model Parallel.
- Distributed ML Training - Training models require a large number of resources over a while and sometimes deploying multiple parallel resources is the only way to bring down training period. The solution supports scheduling and coordinating distributed ML training workloads with dynamic resource allocation
- Distributed ML Serving - Distributed ML serving is a prerequisite in case the model is too large to fit into single GPU memory.The solution facilitates the distribution of interface workloads across multiple nodes, supporting tasks such as large language models (LLMs).
Intelligent Data Federation
Intelligent data federation involves support of data pipelines to ingest data from various data sources, data of various types and formats, processing and storing them across structured, columnar, vector, or file stores.
- Data Pipelines - Support batch and streaming data pipelines for data ingestion, transformation, and storage into appropriate data federation components.
- Data Connectors - Supports ingestion of data from different sources like FTP/SFTP servers, S3 Buckets, Google Drive, Web Pages, etc. Data from different formats is converted into a unified format and stored in respective data store
- Data Processing - Data extracted during the pipeline ingestion process is pre-processed, cleaned, transformed, and then stored. Solution support repository management for distributed file storage and retrieval.
- Data Storage - Solution supports data lake with support of structured, columnar, vector, and file/object storage capabilities. Different storage categories are useful in different applications.
Mega Model Store
Machine learning algorithms are incredibly varied and extensive, each designed to tackle specific problems. Some algorithms are better suited for large datasets with many features, while others excel with smaller or less structured data. The extensive range of algorithms offers flexibility in solving diverse real-world challenges, from predictions and pattern recognition to autonomous decision-making.
- Supervised Models - Supervised learning algorithms learn from labeled data, the solution supports a wide range of supervised and unbalanced supervised models like Decision Trees, Logistic Regression, Neural Networks, etc.
- Time Series Models - Time series algorithms handle time varied data. The solution supports both statistical and ML-based TS models like ARIMA, Prophet, Deep AR, etc.
- Reinforcement Models - The reinforcement learning algorithm focuses on an agent continuously learning in an environment to maximize cumulative rewards. The solution supports multiple reinforcement algorithms like Deep QN, Soft Actor-Critic, etc.
- Other Models - Supports multiple other model categories like Causal and Interpretation, Anomaly Detection, Text LLM, Vision Models etc.
GenAI RAG Engine
Generative AI refers to algorithms that can generate new content, such as text, images, music, or code. Generative AI works on foundation models (LLM) which are built using public data. Retrieval Augmented Generation (RAG) is an approach for building LLM applications, that extends the capabilities of foundation models by providing new or proprietary context information.
- Production-ready RAG Pipeline - Solution supports production-ready RAG pipeline that enables building and deploying LLM applications. Components of the RAG pipeline include Query Rewriter, Chunk Reranker, Retriever, Response Generator, and Response Synthesizer.
- Open Source LLM Models - Solution supports the local deployment of open source LLM models that enable both cost control and enterprise confidentiality.
- Vector DB Integration - Vector databases support chunk storage and retrieval using semantic, keyword, and hybrid search. The solution supports pre-integration with leading vector databases and data pipelines for document ingestion.
- Guardrail Support - Guardrails are becoming important components for securing LLM applications and support configurable policies for text validation and prevent hallucinations
Diverse Deployment Options
Solution offers a range of diverse cloud options tailored to different business needs, including private cloud, public cloud, edge cloud, and high-performance computing (HPC) clusters.
- AI Workload Deployment & Monitor - The solution enables the definition and configuration of clusters, machines, and AI Workloads. It controls Ansible-based deployment of AI Workloads across different machines in the cluster.
- Workload Manager Integration – The solution integrates with industry-leading workload managers (e.g., Slurm), enabling hardware resource allocation, monitoring, and auditing for various model training and inference tasks.
- GPU Acceleration Support - The solution has built-in support for using NVIDIA CUDA and cuDNN software for training and NVIDIA TensorRT for inference activities, enabling the solution components to utilize the available GPU infrastructure.