Back to top

Natural Language Processing - Methods and Applications

Last modified by Anum Afzal Jul 19
   No tags assigned

People in organizations, society and institutions communicate via natural language. Typically, large amounts of unstructured information are stored in text documents. However, it is very challenging for machines to query relevant content fast and extract structured information.

Machine-based solutions for text analysis, indexing and generation are often summarized under the terms natural language processing, text mining or natural language generation. In the past decades, many different methods and solutions have been developed. The diversity of the different problems and the fast development of new technologies, e.g. machine learning, underline the large potential of these methods. 

The seminar aims to give an overview over the technological building blocks and their applications. The participants independently work on a scientific topic, based on existing literature and present and discuss their findings in a presentation and a seminar paper.

Proposed Seminar Topics:

Foundations of NLP:

  1. Word Embeddings: Techniques and Applications
  2. Language Models: From N-grams to Transformer Architectures
  3. Attention Mechanisms in NLP: Foundations and Innovations -

Techniques in NLP:

  1. Document Classification and Topic Modeling
  2. Named Entity Recognition (NER) in Natural Language Processing
  3. Part-of-Speech Tagging and Dependency Parsing: Models, Methods, Evaluation and Applications
  4. Machine Translation: Approaches and Evaluations

Large Language Models:

  1. Transfer Learning and Domain Adaptations: Challenges and Solutions 
  2. Natural Language Generation (Auto Regressive Models): Techniques and Use Cases 
  3. Prompt-Tuning 
  4. Finetuning LLMs and Reinforcement Learning from Human Feedback 
  5. Efficiency and Context Window in Large Language Models 
  6. Text Summarization: Extractive and Abstractive Approaches 
  7. Question Answering Systems: Challenges and Approaches 
  8. Model Hallucination

Conversational AI:

  1. Task-based & Social Conversational Agents 
  2. Dialogue Management (Dialogue State Tracking & Policy) 
  3. Conversational Search Systems 

Privacy & Security in Natural Language Processing:

  1. Ethical Societal, and Legal Aspects of Large Language Models 
  2. Differential Privacy in Natural Language Processing 
  3. Adversarial Attacks in and Privacy Risks of (Large) Language Models 

Miscellaneous:

  1. Natural Language Inference 
  2. Explainability in NLP 
  3. Knowledge Graphs in NLP: Construction and Applications 

Weekly Sessions

Session Date Topic
-1

30.01.2024

14:00 - 15:00

Preliminary Meeting
1

19.04.2024

10 am - 12 pm

1) Introduction 

2)  Word Embeddings: Techniques and Applications

3) Language Models: From N-grams to Transformer Architectures

2

26.04.2024

10 am - 12 pm

1) Attention Mechanisms in NLP: Foundations and Innovations

2) Document Classification and Topic Modeling

3

03.05.2024

10 am - 12 pm

1) Named Entity Recognition (NER) in Natural Language Processing

4

10.05.2024

10 am - 12 pm

no session

5

17.05.2024

10 am - 12 pm

1) Machine Translation: Approaches and Evaluations

2) Transfer Learning and Domain Adaptations: Challenges and Solutions

6

24.05.2024

10 am - 12 pm

1) Natural Language Generation (Auto Regressive Models): Techniques and Use Cases

2) Prompt-Tuning

7

31.05.2024

10 am - 12 pm

no session

8

07.06.2024

10 am - 12 pm

1) Finetuning LLMs and Reinforcement Learning from Human Feedback

2) Efficiency and Context Window in Large Language Models

9

14.06.2024

10 am - 12 pm

1) Machine Translation: Approaches and Evaluations

2) Text Summarization: Extractive and Abstractive Approaches  

10  

21.06.2024

10 am - 12 pm

1) Task-based & Social Conversational Agents

2) Question Answering Systems: Challenges and Approaches 

11  

28.06.2024

10 am - 12 pm

1) Dialogue Management (Dialogue State Tracking & Policy

2) Conversational Search Systems

2)  Part-of-Speech Tagging and Dependency Parsing: Models, Methods, Evaluation and Applications

3) Model Hallucination

 

12  

05.07.2024

10 am - 12 pm

1) Ethical Societal, and Legal Aspects of Large Language Models

2) Differential Privacy in Natural Language Processing

13

12.07.2024

10 am - 12 pm

1) Adversarial Attacks in and Privacy Risks of (Large) Language Models

2) Natural Language Inference

14

19.07.2024

10 am - 12 pm

1) Explainability in NLP

2) Knowledge Graphs in NLP: Construction and Applications

 


 

Deliverables

Prerequisites

  • Regular attendance (not more than one missed session)
  • Active Participation during sessions

Presentations

  • 40 min:
    • 30 min presentation
    • 15 min discussion

Project/Demo

 optional. 0.3 grade bonus

Seminar Paper

  • 8 pages
  • Latex-Template provided on Moodle

Peer Review

  • 2 reviews of other seminar papers

Submissions

Deliverable

Deadline

Format

Final presentation slides

Before your talk

Powerpoint, Keynote or PDF

Code for the project

After your talk

.zip

Seminar paper for peer review

28.07.24

PDF based on provided LaTex template

Peer review

04.08.24

txt-File

Revised seminar paper

11.08.24

PDF based on provided LaTex template

Files and Subpages

Name Type Size Last Modification Last Editor
240130 Afzal NLP Seminar Kickoff.pdf 1,16 MB 30.01.2024 Anum Afzal