Technical Foundations of Generative AI
Learn the technical foundations of generative AI, from neural networks and deep learning to the transformer architectures behind today's large language models.
Modules/Weeks
Weekly Effort
Discipline
School
Format
Cost
Course Description
Modern generative AI is built on decades of advances in machine learning, deep learning, and neural network research. While today's large language models can generate text, answer questions, and power a growing range of AI applications, understanding how they work requires a solid foundation in the technologies that made them possible.
In this course, you'll explore the technical foundations of generative AI, beginning with the mechanics of neural networks and deep learning before progressing through recurrent neural networks (RNNs), long short-term memory (LSTM) networks, and the attention mechanisms and transformer architectures that power today's most advanced AI models.
Through conceptual explanations, hands-on coding exercises using TensorFlow, Keras, and Google Colab, and practical examples, you'll develop a strong technical understanding of the techniques and architectures that underpin modern generative AI.
This course is ideal for:
- Professionals with basic Python programming knowledge who want to deepen their understanding of AI.
- Software developers and other technical professionals interested in the foundations of generative AI.
- Data scientists, machine learning practitioners, and AI professionals looking to strengthen their understanding of modern deep learning architectures.
- Learners who want to understand how today's large language models work beyond simply using AI tools.
Whether you're building AI applications, expanding your machine learning expertise, or seeking a deeper technical understanding of generative AI, this course equips you with the knowledge to understand how modern generative AI works and the architectures that power today's large language models.
Course Prerequisites
To get the most from this course, learners should have:
- Basic Python programming knowledge.
- Familiarity with fundamental programming concepts such as variables, functions, and loops.
- An interest in machine learning and artificial intelligence. No prior experience with neural networks or generative AI is required.
What You Will Learn
This course provides a technical foundation in the concepts and architectures that underpin modern generative AI. You'll learn how neural networks process information, how deep learning architectures evolved to address increasingly complex challenges, and how attention mechanisms and transformer architectures enabled the development of today's large language models. Along the way, you'll explore key concepts through conceptual explanations, practical examples, and hands-on coding exercises using TensorFlow and Keras in Google Colab.
By the end of the course, you will be able to:
- Explain the core mechanics of neural networks, including activation functions, backpropagation, and the vanishing gradient problem.
- Build and evaluate simple neural network models using Python libraries such as TensorFlow and Keras.
- Understand how language models evolved from recurrent neural networks (RNNs) and long short-term memory (LSTM) networks to attention-based transformer architectures.
- Explain how embeddings, attention mechanisms, and transformers enable modern generative AI and large language models.
- Module 1: Introduction to Neural Networks
Build a strong foundation in the core principles of neural networks and deep learning.
- Explore the structure and components of neural networks.
- Learn how activation functions and backpropagation enable neural network training.
- Examine the vanishing gradient problem and the role of ReLU activation.
- Build and evaluate simple neural networks using TensorFlow and Keras.
- Module 2: The Road to Large Language Models (LLMs)
Discover how advances in neural networks and sequence modeling paved the way for today's large language models.
- Understand text analytics concepts, including tokens, vocabularies, and word embeddings.
- Learn how recurrent neural networks (RNNs) process sequential data.
- Explore the limitations of RNNs and how LSTM networks address long-range dependencies.
- Examine how these architectures paved the way for today's large language models.
- Module 3: Attention & Transformers
Learn how attention mechanisms and transformer architectures enabled today's generative AI.
- Understand positional encoding and self-attention.
- Explore multi-head attention and the encoder-decoder transformer architecture.
- Learn how transformers process language more effectively than earlier sequence models.
- Examine how transformer models generate text and enable today's large language models.
Instructors
Johar received an M.A. in Economics from the Birla Institute of Technology and Science and is a Fellow of the Indian Institute of Management Calcutta. He received a Ph.D. in Information Systems from the Stern School of Business at New York University in 1994. Prior to joining Columbia, Johar worked as a quantitative trader at Morgan Stanley, Credit Suisse, and Deutsche Bank, at a tech startup (MSpoke), and taught at the NYU Stern School of Business and the Gabelli School of Business at Fordham University.