Cloud computing systems today, whether open-source or used inside companies, are built using a common set of core techniques, algorithms, and design philosophies – all centered around distributed systems. Learn about such fundamental distributed computing "concepts" for cloud computing.

Some of these concepts include: clouds, MapReduce, key-value/NoSQL stores, classical distributed algorithms, widely-used distributed algorithms, scalability, trending areas, and much, much more!

Know how these systems work from the inside out. Get your hands dirty using these concepts with provided homework exercises. In the programming assignments, implement some of these concepts in template code (programs) provided in the C++ programming language. Prior experience with C++ is required.

The course also features interviews with leading researchers and managers, from both industry and academia.

In the cloud networking course, we will see what the network needs to do to enable cloud computing. We will explore current practice by talking to leading industry experts, as well as looking into interesting new research that might shape the cloud network’s future.

This course will allow us to explore in-depth the challenges for cloud networking—how do we build a network infrastructure that provides the agility to deploy virtual networks on a shared infrastructure, that enables both efficient transfer of big data and low latency communication, and that enables applications to be federated across countries and continents? Examining how these objectives are met will set the stage for the rest of the course.

This course places an emphasis on both operations and design rationale—i.e., how things work and why they were designed this way. We're excited to start the course with you and take a look inside what has become the critical communications infrastructure for many applications today.

Learn the general concepts of data mining along with basic methodologies and applications. Then dive into one subfield in data mining: pattern discovery. Learn in-depth concepts, methods, and applications of pattern discovery in data mining. We will also introduce methods for data-driven phrase mining and some interesting applications of pattern discovery. This course provides you the opportunity to learn skills and content to practice and engage in scalable pattern discovery methods on massive transactional data, discuss pattern evaluation measures, and study methods for mining diverse kinds of patterns, sequential patterns, and sub-graph patterns.

Recent years have seen a dramatic growth of natural language text data, including web pages, news articles, scientific literature, emails, enterprise documents, and social media such as blog articles, forum posts, product reviews, and tweets. Text data are unique in that they are usually generated directly by humans rather than a computer system or sensors, and are thus especially valuable for discovering knowledge about people’s opinions and preferences, in addition to many other kinds of knowledge that we encode in text.

This course will cover search engine technologies, which play an important role in any data mining applications involving text data for two reasons. First, while the raw data may be large for any particular problem, it is often a relatively small subset of the data that are relevant, and a search engine is an essential tool for quickly discovering a small subset of relevant text data in a large text collection. Second, search engines are needed to help analysts interpret any patterns discovered in the data by allowing them to examine the relevant original text data to make sense of any discovered pattern. You will learn the basic concepts, principles, and the major techniques in text retrieval, which is the underlying science of search engines.

Discover the basic concepts of cluster analysis, and then study a set of typical clustering methodologies, algorithms, and applications. This includes partitioning methods such as k-means, hierarchical methods such as BIRCH, and density-based methods such as DBSCAN/OPTICS. Moreover, learn methods for clustering validation and evaluation of clustering quality. Finally, see examples of cluster analysis in applications.

This course will cover the major techniques for mining and analyzing text data to discover interesting patterns, extract useful knowledge, and support decision making, with an emphasis on statistical approaches that can be generally applied to arbitrary text data in any natural language with no or minimum human effort.

Detailed analysis of text data requires understanding of natural language text, which is known to be a difficult task for computers. However, a number of statistical approaches have been shown to work well for the "shallow" but robust analysis of text data for pattern finding and knowledge discovery. You will learn the basic concepts, principles, and major algorithms in text mining and their potential applications.

Learn the general concepts of data mining along with basic methodologies and applications. Then dive into one subfield in data mining: pattern discovery. Learn in-depth concepts, methods, and applications of pattern discovery in data mining. We will also introduce methods for pattern-based classification and some interesting applications of pattern discovery. This course provides you the opportunity to learn skills and content to practice and engage in scalable pattern discovery methods on massive transactional data, discuss pattern evaluation measures, and study methods for mining diverse kinds of patterns, sequential patterns, and sub-graph patterns.

Welcome to the Cloud Computing Applications course, the first part of a two-course series designed to give you a comprehensive view on the world of Cloud Computing and Big Data!

In this first course we cover a multitude of technologies that comprise the modern concept of cloud computing. Cloud computing is an information technology revolution that has just started to impact many enterprise computing systems in major ways, and it will change the face of computing in the years to come.

We start the first week by introducing some major concepts in cloud computing, the economics foundations of it and we introduce the concept of big data. We also cover the concept of software defined architectures, and how virtualization results in cloud infrastructure and how cloud service providers organize their offerings. In week two, we cover virtualization and containers with deeper focus, including lectures on Docker, JVM and Kubernates. We finish up week two by comparing the infrastructure as a service offering by the big three: Amazon, Google and Microsoft.
Week three moves to higher level of cloud offering, including platform as a service, mobile backend as a service and even serverless architectures. We also talk about some of the cloud middleware technologies that are fundamental to cloud based applications such as RPC and REST, JSON and load balancing. Week three also covers metal as a service (MaaS), where physical machines are provisioned in a cloud environment.
Week four introduces higher level cloud services with special focus on cloud storage services. We introduce Hive, HDFS and Ceph as pure Big Data Storage and file systems, and move on to cloud object storage systems, virtual hard drives and virtual archival storage options. As discussion on Dropbox cloud solution wraps up week 4 and the course.

Cloud computing systems today, whether open-source or used inside companies, are built using a common set of core techniques, algorithms, and design philosophies – all centered around distributed systems. Learn about such fundamental distributed computing "concepts" for cloud computing.

Some of these concepts include: clouds, MapReduce, key-value/NoSQL stores, classical distributed algorithms, widely-used distributed algorithms, scalability, trending areas, and much, much more!

Know how these systems work from the inside out. Get your hands dirty using these concepts with provided homework exercises. In the programming assignments, implement some of these concepts in template code (programs) provided in the C++ programming language. Prior experience with C++ is required.

The course also features interviews with leading researchers and managers, from both industry and academia.

This course builds on the material covered in the Cloud Computing Concepts, Part 1 course.

This six-week long Capstone course of the Data Mining Specialization will allow you to apply the learned algorithms and techniques for data mining from the previous courses in the Specialization, including Pattern Discovery, Clustering, Text Retrieval, Text Mining, and Visualization, to solve interesting real-world data mining challenges. Specifically, you will work on a restaurant review data set from Yelp and use all the knowledge and skills you’ve learned from the previous courses to mine this data set to discover interesting and useful knowledge. The design of the Capstone emphasizes: 1) simulating the workflow of a data miner in a real job setting; 2) integrating different mining techniques covered in multiple individual courses; 3) experimenting with different ways to solve a problem to deepen your understanding of techniques; and 4) allowing you to propose and explore your own ideas creatively.

The goal of the Capstone project is to analyze and mine a large Yelp review data set to discover useful knowledge to help people make decisions in dining. The project will include the following outputs:

1. Opinion visualization: explore and visualize the review content to understand what people have said in those reviews.
2. Cuisine map construction: mine the data set to understand the landscape of different types of cuisines and their similarities.
3. Discovery of popular dishes for a cuisine: mine the data set to discover the common/popular dishes of a particular cuisine.
4. Recommendation of restaurants to help people decide where to dine: mine the data set to rank restaurants for a specific dish and predict the hygiene condition of a restaurant.

From the perspective of users, a cuisine map can help them understand what cuisines are there and see the big picture of all kinds of cuisines and their relations. Once they decide what cuisine to try, they would be interested in knowing what the popular dishes of that cuisine are and decide what dishes to have. Finally, they will need to choose a restaurant. Thus, recommending restaurants based on a particular dish would be useful. Moreover, predicting the hygiene condition of a restaurant would also be helpful.

By working on these tasks, you will gain experience with a typical workflow in data mining that includes data preprocessing, data exploration, data analysis, improvement of analysis methods, and presentation of results. You will have an opportunity to combine multiple algorithms from different courses to complete a relatively complicated mining task and experiment with different ways to solve a problem to understand the best way to solve it. We will suggest specific approaches, but you are highly encouraged to explore your own ideas since open exploration is, by design, a goal of the Capstone.

You are required to submit a brief report for each of the tasks for peer grading. A final consolidated report is also required, which will be peer-graded, and the top 20 reports, as evaluated by peer review, will also be judged by an Industry Expert Committee who will select 10 of the reports as the Best Capstone Projects. The Industry Expert Committee is composed of multiple relevant industry experts representing different companies.

Note: You should complete all the other courses in this Specialization before beginning this course.

This six-week long Project course of the Cloud Computing Specialization will allow you to apply the learned theories and techniques for cloud computing from the previous courses in the Specialization, including Cloud Computing Concepts, Part 1, Cloud Computing Concepts, Part 2, Cloud Computing Applications, Part 1, Cloud Computing Concepts, Part 2, and Cloud Networking.

Note: You should complete all the other courses in this Specialization before beginning this course.

This six-week long Project course of the Data Mining Specialization will allow you to apply the learned algorithms and techniques for data mining from the previous courses in the Specialization, including Pattern Discovery, Clustering, Text Retrieval, Text Mining, and Visualization, to solve interesting real-world data mining challenges. Specifically, you will work on a restaurant review data set from Yelp and use all the knowledge and skills you’ve learned from the previous courses to mine this data set to discover interesting and useful knowledge. The design of the Project emphasizes: 1) simulating the workflow of a data miner in a real job setting; 2) integrating different mining techniques covered in multiple individual courses; 3) experimenting with different ways to solve a problem to deepen your understanding of techniques; and 4) allowing you to propose and explore your own ideas creatively.

The goal of the Project is to analyze and mine a large Yelp review data set to discover useful knowledge to help people make decisions in dining. The project will include the following outputs:

1. Opinion visualization: explore and visualize the review content to understand what people have said in those reviews.
2. Cuisine map construction: mine the data set to understand the landscape of different types of cuisines and their similarities.
3. Discovery of popular dishes for a cuisine: mine the data set to discover the common/popular dishes of a particular cuisine.
4. Recommendation of restaurants to help people decide where to dine: mine the data set to rank restaurants for a specific dish and predict the hygiene condition of a restaurant.

From the perspective of users, a cuisine map can help them understand what cuisines are there and see the big picture of all kinds of cuisines and their relations. Once they decide what cuisine to try, they would be interested in knowing what the popular dishes of that cuisine are and decide what dishes to have. Finally, they will need to choose a restaurant. Thus, recommending restaurants based on a particular dish would be useful. Moreover, predicting the hygiene condition of a restaurant would also be helpful.

By working on these tasks, you will gain experience with a typical workflow in data mining that includes data preprocessing, data exploration, data analysis, improvement of analysis methods, and presentation of results. You will have an opportunity to combine multiple algorithms from different courses to complete a relatively complicated mining task and experiment with different ways to solve a problem to understand the best way to solve it. We will suggest specific approaches, but you are highly encouraged to explore your own ideas since open exploration is, by design, a goal of the Project.

You are required to submit a brief report for each of the tasks for peer grading. A final consolidated report is also required, which will be peer-graded.

Welcome to the Cloud Computing Applications course, the second part of a two-course series designed to give you a comprehensive view on the world of Cloud Computing and Big Data!

In this second course we continue Cloud Computing Applications by exploring how the Cloud opens up data analytics of huge volumes of data that are static or streamed at high velocity and represent an enormous variety of information. Cloud applications and data analytics represent a disruptive change in the ways that society is informed by, and uses information. We start the first week by introducing some major systems for data analysis including Spark and the major frameworks and distributions of analytics applications including Hortonworks, Cloudera, and MapR. By the middle of week one we introduce the HDFS distributed and robust file system that is used in many applications like Hadoop and finish week one by exploring the powerful MapReduce programming model and how distributed operating systems like YARN and Mesos support a flexible and scalable environment for Big Data analytics. In week two, our course introduces large scale data storage and the difficulties and problems of consensus in enormous stores that use quantities of processors, memories and disks. We discuss eventual consistency, ACID, and BASE and the consensus algorithms used in data centers including Paxos and Zookeeper. Our course presents Distributed Key-Value Stores and in memory databases like Redis used in data centers for performance. Next we present NOSQL Databases. We visit HBase, the scalable, low latency database that supports database operations in applications that use Hadoop. Then again we show how Spark SQL can program SQL queries on huge data. We finish up week two with a presentation on Distributed Publish/Subscribe systems using Kafka, a distributed log messaging system that is finding wide use in connecting Big Data and streaming applications together to form complex systems. Week three moves to fast data real-time streaming and introduces Storm technology that is used widely in industries such as Yahoo. We continue with Spark Streaming, Lambda and Kappa architectures, and a presentation of the Streaming Ecosystem. Week four focuses on Graph Processing, Machine Learning, and Deep Learning. We introduce the ideas of graph processing and present Pregel, Giraph, and Spark GraphX. Then we move to machine learning with examples from Mahout and Spark. Kmeans, Naive Bayes, and fpm are given as examples. Spark ML and Mllib continue the theme of programmability and application construction. The last topic we cover in week four introduces Deep Learning technologies including Theano, Tensor Flow, CNTK, MXnet, and Caffe on Spark.

The Internet of Things (IoT) stands to be the next revolution in computing. Billions of data-spouting devices connected to the Internet are already fundamentally changing the way we live and work. This course teaches a deep understanding of IoT technologies from the ground up. Students will learn IoT device programming (Arduino and Raspberry Pi), sensing and actuating technologies, IoT protocol stacks (Zigbee, 5G, NFC, MQTT, etc), networking backhaul design and security enforcement, data science for IoT, and cloud-based IoT platforms such as AWS IoT. As an optional honors avtivity, students will be guided through laboratory assignments designed to give them practical real-world experience, where they will deploy a distributed wifi monitoring service, a cloud-based IoT service platform serving tens of thousands of heartbeat sensors, and more. Students will emerge from the class with a cutting-edge education on this rapidly emerging technology segment, and with the confidence to carry out tasks they will commonly encounter in industrial settings. Important: To complete the practical part of the whole series (honors) there will be practical experimentation using actual hardware, which you will need to acquire. (Cost may vary between 100 and 200 USD depending on your location). Most parts that are needed for the first course, will be re-used in the following courses.

This course builds on the first two courses in this series: IoT Devices and IoT Communications. Here you will begin to learn enterprise IoT. Enterprise networks, from first-hop access to backend IoT services are critical because they allow your IoT devices to reach the Internet and achieve their true intelligence. IoT places extreme demands on first-hop access - ultra-dense deployments challenge spectrum allocation, the need to provide strong segmentation yet let devices reach into IoT services such as gateways and databases. During this week you will begin to learn about these challenges, and the underlying protocols and technologies of wired networks that can help you to address them.

This course is the last course in our series of four courses and builds on the previous three courses: IoT Devices, IoT Communications, and IoT Networking. After we have built and programmed a small self-driving vehicle, we then set out to enhance its connectivity and add important security infrastructure. In this course we will now look closer into various remaining types of decentralized network topography. In the lab, we will additionally cover important cloud technologies based on machine-learning. In the first two weeks' lectures, we will cover important components of networks. Metaphorically speaking, when you learn how the human body works, you start by understanding the "organs", the stomach, the liver, and so on. Likewise, we can best understand networks by understanding the individual components that make them up and their function. In this lecture series we will study "devices" such as routers, switches, firewalls, load balancers, and many more. We will learn about how they individually operate, how they are configured, and how they work together to achieve various network-wide properties and goals.