Do you want to go beyond being a LLM user and learn how to build and manipulate them from scratch? Do you want to build your own small and efficient custom models? This is the course for you.

Deep Learning neural network models have been successfully applied to natural language processing, and have changed radically how we interact with machines, including generating content, searching and processing information. These models are able to infer a continuous representation for words and documents, and generalize to new tasks with much less training data than classical machine learning algorithms.

The seminar will introduce the main deep learning models used in natural language processing, allowing the attendees to gain hands-on understanding and implementation of them in Keras.

This 20 hour introduction covers the latest developments, including Transformers and pre-trained (multilingual) language models that underlie GPT, as well as how to fine-tune them, train them to follow instructions or learn from human feedback and verifiable rewards. It combines theoretical and practical hands-on classes. Attendants will be able to understand and implement the models in Keras.

The course is part of the NLP master hosted by the Ixa NLP research group at the HiTZ research center of the University of the Basque Country (UPV/EHU).

Student profile

Addressed to professionals, researchers and students who want to understand and apply deep learning techniques to text. The practical part requires basic programming experience, a university-level course in computer science and experience in Python. Basic math skills (algebra or pre-calculus) are also needed.

Contents

Introduction to machine learning and NLP with Keras

Machine learning, Deep learning
Natural Language Processing
A sample NLP task with ML
. Sentiment analysis
. Features
. Logistic Regression
LABORATORY: Sentiment analysis with logistic regression

Multilayer Perceptron and Word Embeddings

Multiple layers ~ Deep: MLP
Backpropagation and gradients
Learning rate
More regularization
Hyperparameters
Representation learning
Word embeddings
LABORATORY: Sentiment analysis with Multilayer Perceptron

Recurrent Neural Networks, Seq2seq, Neural Machine Translation

From words to sequences: RNNs
. Language Models (sentence encoders)
. Language Generation (sentence decoders)
. Sequence to sequence models and Neural Machine Translation (I)
Better RNNs: LSTM
LABORATORY: Sentiment analysis with LSTMs

Attention, Transformers and Document Representations

Attention and memory
Transformers and self-attention
Evaluating Document Representations
LABORATORY: Attention Model for Document Representations

Large Language Models

Pre-training
Fine-tuning, Prompting, Instructions
Reinforcement Learning, Preferences, Human Feedback
Test-time-inference, Thinking and Verifiable Rewards
Multilinguality
LABORATORY: Pre-trained transformers for sentiment analysis and NLI

Instructors

Person 1

Eneko Agirre

Professor, member of IXA
Director of HiTZ
ACL fellow, Spanish CS Research Award

Person 2

Imanol Miranda

PhD researcher
member of IXA
and HiTZ

Practical details

General information

Part of the Language Analysis and Processing master program.
  • The classes will be broadcasted live online. The practical labs will be also held online tutorized by a lecturer.
  • 5 theoretical sessions with interleaved hands-on labs (20 hours).
  • Scheduled from June 1st to 5th, 2026, 15:00-19:00 CET.
  • Teaching language: English.
  • Capacity: 60 attendants (First-come first-served).
  • Cost: 400€ + 4€ insurance = 404€
    (If you are an UPV/EHU member or have already registered for another course, it is 400€).

Registration

If you are interested in any of the courses, Fill out the form

Early editions

Covid times: Online class of July 2020, with a handful of the 70 participants.

Class of January 2020

Class of July 2019