Heartbreak Motel

Anecdotal Bigdata Ecosphere

"Sometimes maybe good sometimes maybe shit."

本文转载自如何用形象的比喻描述大数据的技术生态？Hadoop、Hive、Spark 之间是什么关系？如何用形象的比喻描述大数据的技术生态？Hadoop、Hive、Spark 之间是什么关系？ Xiaoyu Ma的回答作者：Xiaoyu Ma 链接：https://www.zhihu.com/question/27974418/answer/389657...

Posted by Mugen on March 9, 2019

Difference between Distributed and Cluster?

"body, head, limbs VS two men"

本文转载自分布式与集群的区别是什么？1 2 分布式与集群的区别是什么？知乎用户从三种结构上来讲作者：大闲人柴毛毛链接：https://www.zhihu.com/question/20004877/answer/282033178 来源：知乎著作权归作者所有。商业转载请联系作者获得授权，非商业转载请注明出处。三种结构： ...

Posted by Mugen on March 9, 2019

How to use Spark?

"Take it easy."

心急吃不了热豆腐原来学使用也可以从入门到放弃第一步在看： A Gentle Introduction to Apache Spark on Databricks 哇，好长不看，去了下一个链接： Spark ALS algorithm ML Based 直接又是扔给我一堆代码…… 在了解啥是ALS之前先看看RDD的API吧…… Collaborative Filte...

Posted by Mugen on March 9, 2019

What is Apache Spark?

"an open-source distributed general-purpose cluster-computing framework"

本文转载自 Spark简介(Python版) Spark简介 Spark 在 2009 年左右诞生于加州大学伯克利分校的著名 AMPLab。最初推动 Spark 成名的原因是它能够经常在内存执行大量的计算工作，直到作业的最后一步才写入磁盘。工程师通过弹性分布式数据集（RDD）理念实现了这一目标，在底层 Pipeline 中能够获取每个阶段数据结果的所有派生关系，并且允许在机器故障...

Posted by Mugen on March 9, 2019

Introduction to RDD(Resilient Distributed Datasets)

"Just a fundamental data structure of Spark."

本文摘自 Spark入门：RDD的设计与运行原理(Python版) RDD的设计与运行原理 Spark的核心是建立在统一的抽象RDD（Resilient Distributed Datasets）之上，使得Spark的各个组件可以无缝进行集成，在同一个应用程序中完成大数据计算任务。RDD的设计理念源自AMP实验室发表的论文《Resilient Distributed Datase...

Posted by Mugen on March 8, 2019

What is the difference between MapReduce and Spark?

"You are Neo, I'm matrix, and sometimes Agent Smith."

本文转载自 MapReduce和Spark的区别是什么？ MapReduce和Spark的区别是什么？知乎用户的回答作者：知乎用户链接：https://www.zhihu.com/question/53354580/answer/307863620 来源：知乎著作权归作者所有。商业转载请联系作者获得授权，非商业转载请注明出处。首先大...

Posted by Mugen on March 8, 2019

What is Hadoop?

"From Google Cloud to Hadoop"

From the beginning to the end. Hadoop是Google的云计算系统的Java开源实现，是一种分布式系统的平台，通过它可以很轻松的搭建一个高效、高质量的分布式系统。谷歌集群系统主要包括三个部分（所谓的Google三驾马车）：分布式文件系统GFS The Google File System发表于SOSP 19(Symposi...

Posted by Mugen on March 8, 2019

What is MapReduce

"MapReduce is a programming model and an associated implementation for processing and generating big data sets with a parallel, distributed algorithm on a cluster."

本文转载自关于MapReduce的理解？关于MapReduce的理解？刘通的回答 MapReduce是一个分布式编程计算模型，用于大规模数据集的分布式系统计算。 Q:是否可以理解为： Map负责分发任务 Reduce负责回收完成的任务 A:并非分发和回收。而是一种计算思想的2个阶段。 Map负责计算 Reduce负责处理作者：刘通链接...

Posted by Mugen on March 8, 2019

大数据与Spark从入不了门到快入门

"I'm fine. FAQ"

终于得空把上个月26号当天的一系列学习总结发布上来~ Apache 相关大数据相关框架译文一篇：Hadoop、Spark等5种大数据框架对比，你的项目该用哪种？ by 大愚若智译原文在此：Hadoop, Storm, Samza, Spark, and Flink: Big Data Frameworks Compared by Justin Ellingwood ...

Posted by Mugen on March 8, 2019

Begin to use Git

"Using git"

现在的印象笔记有积累，但是没有体系，也就是固化不到自己的体系中，这个是个问题，需要注意一下。也许技术性的东西，不再适合在印象笔记里放了。一开始接触的，但是有些地方语焉不详。本地jekyll还需要参考提到用prose在线编辑 1. 利用github-pages建立个人博客 Including git, 2. Git 安装和使用教程 Theses are useful. ...

Posted by Mugen on February 28, 2019

ABOUT ME

Mugen

Anecdotal Bigdata Ecosphere

"Sometimes maybe good sometimes maybe shit."

Difference between Distributed and Cluster?

"body, head, limbs VS two men"

How to use Spark?

"Take it easy."

What is Apache Spark?

"an open-source distributed general-purpose cluster-computing framework"

Introduction to RDD(Resilient Distributed Datasets)

"Just a fundamental data structure of Spark."

What is the difference between MapReduce and Spark?

"You are Neo, I'm matrix, and sometimes Agent Smith."

What is Hadoop?

"From Google Cloud to Hadoop"

What is MapReduce

"MapReduce is a programming model and an associated implementation for processing and generating big data sets with a parallel, distributed algorithm on a cluster."

大数据与Spark从入不了门到快入门

"I'm fine. FAQ"

Begin to use Git

"Using git"

FEATURED TAGS

FRIENDS