The information present in Web and multimodal data allow the creation of systems capable of answering diverse and complex user information needs, such as in the medical domain or in the social media domain. The objective of this course is to allow students to understand all aspects related the representation, extraction and access to Web and multimodal information. Students will learn the main techniques to create multimodal search spaces, indexing through similarity-based hashing and annotation of information to allow advanced search.

A project will be developed throughout the semester, allowing the student to consolidate the techniques studied into a single Web/multimodal search system targeting a specific information domain, e.g., medical, video surveillance and social media.

Knowledge:
- Understand the concept of error in information representation.
- Understand Web and multimedia representation models.
- Analyze Web and multimedia data for information extraction.
- Understand Web and multimedia information access paradigms.

Know-how:
- Implement Web and multimedia information representation algorithms.
- Implement Web and media information annotation algorithms.
- Develop Web and multimedia information access systems.
- Understand domain specific information needs.

Soft-Skills:
- Design Web and multimodal information retrieval systems.
- Select the right techniques do solve problems dealing with multimodal information.
- Be able to design experimental protocols and analyze experimental results.

1. Introduction
2. Social-media data representations
3. Social-networks analysis
4. Multimodal information extraction
5. Learning to rank
6. Case study
7. Distributed representations of words
8. Multimodal embeddings
9. Tagging social-media information
10. Near-duplicate detection
11. Browsing similar documents

[1] Bing Liu, “Web Data Mining: Exploring Hyperlinks, Contents, and Usage Data”, 2013
[2] Mathias Lux, Oge Marques, “Visual Information Retrieval using Java and LIRE”, 2013
[3] Rick Szeliski , “Computer Vision: Algorithms and Applications”, 2013