Big Data - Survival Guide!


Created by Luan Cestari / G+/ Facebook/ @BR_LuanCestari

Who is the presenter?

Luan = algorithms + entrepreneur + computer nerd/geek + open source


Who are you?

Are you a boy or a girl?

Doctor Oak from pokemon


  • History
  • Introduction to Big Data
  • Lambda Architecture
  • Solutions

Key points

  • Business/Job Opportunities
  • Architecture
  • Technologies

What's Big Data

What's Big Data

4 Vs

  • Volume
  • Variety
  • Velocity
  • Value

Whats size of Big Data?

What is the size of daily job from Facebook? 100 GB 10000GB 100000GB?

Where does the data come from?

  • Customer generated Content
  • M2M
  • Sensors
  • B2B
  • B2C

Why so much data?

Unstructured data x Structured data


and NOSQL!!

and NewSQL!!

and ...?


  • Started Hierarchical Database in 60`s
  • The ralacional in 80`s
  • Until recently it seems to be the only solution


Map of SQL and related

Lambda Architecture

What usually happens (and goes wrong)

Go Wrong: A Solution that grown

Go Wrong: Wrong Tools/Technoliges

Example of Web Analysis

Facebook case

  • +1 Billion users
  • +240 Billion photos
  • +1 Trillion connections
  • 22% of references of the Internet

What usually happens (and goes wrong)

Batch Layer

Service Layer

Speed Layer

Putting it all together!

Example using JBoss Tusk


BY Luan Cestari