29:52Entity Resolution at Scale • Huon Wilson • YOW! 2019
From the creator
This presentation was recorded at YOW! 2019. #GOTOcon #YOW https://yowcon.com Huon Wilson - Software Engineer at CSIRO's Data61 RESOURCES https://www.linkedin.com/in/huon-wilson https://twitter.com/huon_w https://github.com/huonw https://huonw.github.io ABSTRACT Real world #data is rarely clean: there are often corrupted and duplicate records, and even corrupted records that are duplicates! One step in #DataCleaning is #EntityResolution: connecting all of the duplicate records into the single underlying entity that they represent. This talk will describe how we approach entity resolution, and look at some of the challenges, solutions and lessons learnt when doing entity resolution on top of #ApacheSpark, and scaling it to process billions of records. [...] RECOMMENDED BOOKS Adi Polak • Machine Learning with Apache Spark • https://amzn.to/3ppdUkB Holden Karau & Rachel Warren • High Performance Spark • https://amzn.to/3v2eLbn Holden Karau, Konwinski, Wendell & Zaharia • Learning Spark • https://amzn.to/397e2NE https://twitter.com/GOTOcon https://www.linkedin.com/company/goto- https://www.instagram.com/goto_con https://www.facebook.com/GOTOConferences #DataEngineering #HuonWilson #SoftwareEngineering #Programming #YOWcon Looking for a unique learning experience? Attend the next GOTO conference near you! Get your ticket at https://gotopia.tech Sign up for updates and specials at https://gotopia.tech/newsletter SUBSCRIBE TO OUR CHANNEL - new videos posted almost daily. htt
Published 2023-12-30 · Watch on YouTube ↗
Availability and playback restrictions are controlled by the original channel.
Keep exploring Learn to Code
29:52
04:07
11:04"Anti-Slop" Designs with Impeccable (a Crash Course) #5 - Enable Image Generation
Net Ninja
2:25:40
42:55Java Graphics Programming Tutorial - How To Draw Shapes, Paths, Curves, and Apply Transformations
choobtorials
08:10
18:31