BIG DATA TOOLS - PART 2

Code Cours
2324-IÉSEG-MBD1S2-MIS-MBDCE07UE
Language of instruction
English
Teaching content
MANAGEMENT OF INFORMATION SYSTEMS
This course occurs in the following program(s)
Training officer(s)
S.HOORNAERT
Stakeholder(s)
S.HOORNAERT
Level
-
Program year
Period

Présentation

Prerequisite
- Participants should be familiar with the basic concepts of R (e.g., vectors, dataframes, functions, packages).
- Participants should be familiar with reading and writing SQL queries (e.g., select, group by, having).
- Participants should know the basic concepts of Business Analytics and Predictive Modeling.
Goal
At the end of the course, the student should be able to:
- understand the available technologies in the Big Data universe and use the correct technology for a given Big Data problem
- know the technologies for reading and writing Big Data (e.g., MapReduce, Hadoop, HDFS, Parquet)
- know Spark, its architecture and its APIs
- use Spark as a tool for descriptive and predictive analytics using Spark SQL, MLlib, Streaming, and GraphX
- solve and present an end-to-end solution to a Big Data problem in an intercultural team

These competencies and/or skills contribute to the following learning objectives
- 1.B Successfully collaborate within a intercultural team
- 3.A Breakdown complex organizational problems using the appropriate methodology
- 4.B Compose constructive personal feedback and guidance
- 4.C. Convey powerful messages using contemporary presentation techniques
- 5.B Construct expert knowledge from cutting-edge information
- 7.A Demonstrate an expertise on key concepts, techniques and trends in their professional field
Presentation
Every day, 2.5 quintillion bytes (=2.5*10^18 bytes) of data are created. Every minute, more than 4.2 million posts are liked and 300 hours of videos are uploaded. This generated (Big) data is characterized by its volume, variety, velocity, and veracity, and requires a specific approach for reading, writing, transforming, and modeling. This course introduces the problem of Big Data, the Big Data universe, reading and writing Big Data, and the skills to work with these data. It uses Spark as a core processing engine for running descriptive and predictive analyses on Big Data.

Modalités

Organization
Type Amount of time Comment
Travail personnel
Group Project 28,00
Individual Project 6,00
Charge de travail personnel indicative 34,00
Présentiel
Cours magistral 16,00
Cours interactif 16,00
Overall student workload 100,00
Evaluation
The assessment will consist of:
- a group work where students will solve a Big Data case study end-to-end with group feedback
- a set of individual assignments to support learning Spark with individual feedback
- a written exam to test the knowledge of Big Data, the Big Data universe, and Spark.
Control type Duration Amount Weighting
Autres
Projet Collectif 28,00 0 35,00
Projet Individuel 6,00 1 15,00
Examen (final)
Examen écrit 4,00 1 50,00
TOTAL 100,00

Ressources

Bibliography
Chambers, Bill, and Matei Zaharia. Spark: the definitive guide: big data processing made simple. " O'Reilly Media, Inc.", 2018. -
Internet resources