I have listened many times about “docker”, “container”. On summer 2015, I even attended to a talk of Eric Brewer about Kubernetes. More than one year later, they came to my life. I came to this world with very limited knowledge about container, docker, since my expertise is about big data processing frameworks. After playing around … Continue reading Some first thoughts on using Docker Swarm and Kubernetes
[Tutorial] Deploy a Kubernetes cluster on Ubuntu 16.04
I haven’t written any posts from July since I’ve changed my job and started working on something new. Check my about page for more information. The new work requires me to work with Docker, Docker Swarm, Kubernetes. I’ve worked on Swarm around 2 months before working on Kubernetes. In my opinion, Kubernetes is more mature than … Continue reading [Tutorial] Deploy a Kubernetes cluster on Ubuntu 16.04
[Tutorial]How to use spark-sql-perf
The situation is that: there’s one of our work need to be benchmarked. It’s still the SparkSQL Server that I designed and developed but new sharing technique was implemented and integrated into it. So, we decided to use the spark-sql-perf of Databricks to benchmark our work, and the benchmarked queries we used is TPCDS.
[Implementation]Simultaneous Pipeline technique of MRShare
After 6 months working, I've just defensed my master thesis at the middle of September. The system works properly and efficiently. So sorry since there wasn't any posts during July to August. As the previous posts, I had done with the design of my system, the rest that I need to do is implementing them. The … Continue reading [Implementation]Simultaneous Pipeline technique of MRShare
[SysDeg] Worksharing Framework and its design: Some modifications
After a lot of discussions, we decided to change the design of the system a little bit, so it can be more general and extensible. The new design is described as the figure above. The WorkSharing Detector remains the same as the old design. Its goal is generating bags of DAGs which are labeled with … Continue reading [SysDeg] Worksharing Framework and its design: Some modifications
