AI for Data Teams: LLMs, RAG and Reviewing AI Output
Use Generative AI effectively in data engineering | modeling | analytics and documentation | Explore LLMs | prompt engineering | RAG | vector search and responsible AI practices | Build prompts for SQL generation and data quality rules | Develop RAG pipeline using metadata.
Syllabus · 4 modules · Sat–Sun, 10am–2pm IST
- LLM Basics for Data People · 3h
Introduction to Generative AI and LLMs
Traditional AI versus Generative AI
LLM architecture at a conceptual level
Tokens, tokenization and context windows
Training data, inference and next-token prediction
Model parameters and temperature
System prompts, user prompts and conversation context
Text generation and structured output
Strengths and limitations of LLMs
Hallucinations, outdated knowledge and ambiguous answers
Deterministic versus probabilistic outputs
Common use cases in data engineering, modeling and analytics
Overview of hosted and open-source LLMs
Responsible use of AI in enterprise environments
Ask an LLM to explain a complex SQL query.
Compare responses to the same prompt using different instructions.
Test how ambiguous business requirements affect generated answers.
Identify incorrect SQL and fabricated database objects in AI output.
- Prompting for Data Work · 4h
Prompt engineering fundamentals
Anatomy of a well-structured prompt
Role, task, context, constraints and output format
Zero-shot and few-shot prompting
Prompt templates and reusable prompt libraries
Structured outputs using JSON and tables
Prompting for SQL generation and explanation
Prompting for query debugging and optimization
Generating data dictionaries and technical documentation
Generating source-to-target mapping (STTM) drafts
Business requirement analysis and business rule extraction
AI-assisted entity and attribute identification
Iterative prompt refinement
Common prompt failure patterns
Generate SQL from a business question.
Generate table and column descriptions from database metadata.
Convert a business requirement into candidate entities and relationships.
Generate data quality rules from sample data.
- Retrieval and Vector Search · 4h
Limitations of LLMs when answering enterprise-specific questions
Introduction to Retrieval-Augmented Generation
RAG architecture and components
Document ingestion and preprocessing
Text extraction and cleaning
Chunking strategies and chunk size
Vector databases and vector indexes
Top-K retrieval
Metadata filtering
Context construction and prompt augmentation
RAG response generation
Basic RAG evaluation
Retrieval failures and incomplete context
Split documents into chunks.
Generate embeddings using an available embedding model.
Store and retrieve embeddings using a vector store.
Build a simple question-answering pipeline over metadata.
Compare ordinary LLM answers with RAG-grounded answers.
- Safe use and review · 5h
Responsible AI in data teams
Data privacy and confidentiality
Personally identifiable information (PII)
Sensitive data and enterprise data policies
Risks of sharing proprietary metadata
Safe use of public versus enterprise AI tools
Access control and data minimization
Prompt injection and untrusted retrieved content
Reviewing AI-generated SQL
Reviewing AI-generated data models and documentation
SQL execution, testing and reconciliation
Hallucination detection and factual verification
AI governance and auditability
Limitations of automated evaluation
Capstone project presentation and trainer review
Review an AI-generated SQL query and identify errors.
Identify sensitive information in sample prompts.
Evaluate RAG answers for relevance, correctness and source support.
Design a human approval workflow for AI-generated data artifacts.
Explain how LLMs work, including tokens, context windows, embeddings and model limitations.
Identify hallucinations and other common failure modes in AI-generated content.
Design structured prompts for SQL generation, data documentation and data quality tasks.
Use AI to generate and review SQL queries, transformation logic and source-to-target mapping drafts.
Create prompts that support business requirement analysis and data modeling.
Explain embeddings, chunking, vector databases and semantic search.
Build a basic RAG pipeline over metadata and technical documentation.
Retrieve relevant information and generate context-grounded responses.
Evaluate AI-generated SQL, documentation and metadata using systematic review checklists.
Identify sensitive information and apply appropriate data protection practices.
Establish human review gates for AI-assisted data development.
Deliver a practical AI-assisted data project with documented testing and validation.
