[SysDeg] Worksharing Framework and its design – Part 3: Prototype for the first version

Long time no see! After one month playing with caching in Spark, I learned many valuable lessons (which will be posted on other blog posts, about Cache Manager and Block Manager of Spark). Our team came back to the  design of the system - spark SQL server. To be honest, i spent too much time … Continue reading [SysDeg] Worksharing Framework and its design – Part 3: Prototype for the first version

Advertisements

[Sysdeg] Worksharing framework and its design – Part 1

As the previous post I mentioned, my internship will mostly focus on designing and implementing a worksharing framework for GROUPING SETS on Apache Spark. I will briefly discuss about the concept of the framework and its design in this post. Actually, many sharing (scan, computation) frameworks have been proposed in also traditional database systems and in … Continue reading [Sysdeg] Worksharing framework and its design – Part 1

My internship and some documents on Apache Spark

I heard about what I will do in my internship 6 months ago. Well, to be precise, it was right after I finished my summer internship. It is designing and building a worksharing framework (scan, computation)  for Pig queries - Hadoop MapReduce, which mostly focuses on GROUPING SETS operation. 6 months later, my internship still … Continue reading My internship and some documents on Apache Spark