{"id":13919,"date":"2021-08-31T17:04:31","date_gmt":"2021-08-31T11:34:31","guid":{"rendered":"http:\/\/ismiletechnologies.com\/?p=13919"},"modified":"2022-12-10T01:59:33","modified_gmt":"2022-12-09T20:29:33","slug":"modern-data-platform","status":"publish","type":"post","link":"https:\/\/ismiletechnologies.com\/en_us\/dataops\/modern-data-platform\/","title":{"rendered":"Modern Data Platform"},"content":{"rendered":"\r\n

Due to\u00a0an increase\u00a0in service providers in the cloud computing domain, there is a lot of options available for the organizations to choose a platform for storing their data, processing it, presenting\u00a0it,\u00a0and\u00a0using\u00a0advanced\u00a0technologies. Modern data platform is a\u00a0set of different tools that are\u00a0brought together to manage and use the data efficiently for organizations to take data driven decisions.\u00a0<\/p>\r\n\r\n\r\n\r\n

Modern Data Platform should be made considering that it should be highly scalable, highly available and must be include all the tools required in it.\u00a0The main components of MDP\u00a0are as follows:\u00a0<\/p>\r\n\r\n\r\n\r\n

\"\"<\/figure>\r\n\r\n\r\n\r\n
    \r\n
  1. Ingesting the data- This is the start of the MDP process where you\u00a0must\u00a0consider and select tools that will be used to fetch or access the data from different sources. These tools will pull the data and then\u00a0give it to the\u00a0MDP,\u00a0and it works as a connection between the MDP and external data sources.\u00a0The data coming from the sources may be structured, unstructured, streamlined or batch data.\u00a0<\/li>\r\n<\/ol>\r\n\r\n\r\n\r\n
      \r\n
    1. Data\u00a0Storage- The data retrieved from various sources must be stored by the organization for later usage. For this, we can use Cloud\u00a0Datalake, Cloud data store, data warehouse, etc. This must be selected\u00a0based on\u00a0cost, accessibility, and speed of access.\u00a0<\/li>\r\n<\/ol>\r\n\r\n\r\n\r\n
        \r\n
      1. Data processing- The data stored in the company\u2019s storage is raw data and cannot be directly used for any purpose.\u00a0So,\u00a0the data has to undergo some processing in order to be used for making business decisions and getting any insights from it. For this, we can use Python\/R,\u00a0Bigquery,\u00a0SQL, etc. This will convert raw data into usable data.\u00a0<\/li>\r\n<\/ol>\r\n\r\n\r\n\r\n
          \r\n
        1. Data Modelling and advanced technologies- This data can be used to feed into advanced technological models like machine learning, deep learning, etc.\u00a0We can use Anaconda,\u00a0AWS\u00a0sagemaker, GCP ML engine, etc. These\u00a0models will help in understanding patterns or getting insights from the data.\u00a0<\/li>\r\n<\/ol>\r\n\r\n\r\n\r\n
            \r\n
          1. Data presentation- This is the output stage of MDP. Here we can use tools which\u00a0help\u00a0in\u00a0creating\u00a0views, dashboards, etc.\u00a0Tools like Tableau,\u00a0PowerBI, Google\u00a0analytics,etc\u00a0can be used. This will be used for visualizing the data and works as a connection between MDP and humans to easily\u00a0understand the data and take business\u00a0decisions\u00a0which would be data driven.\u00a0<\/li>\r\n<\/ol>\r\n\r\n\r\n\r\n
              \r\n
            1. Data Privacy, Access control- Data governance is one of the most important\u00a0practices\u00a0that should be implemented due to\u00a0the threat\u00a0of hacking and data being leaked out of organization. For this, we\u00a0can use AWS identity and access\u00a0management, security groups, etc.\u00a0<\/li>\r\n<\/ol>\r\n\r\n\r\n\r\n

              An MDP is a collection of tools that must be carefully selected, and the bundle of tools must be enough for everything that needs to maintain and use the data for the organization.\u00a0<\/p>\r\n","protected":false},"excerpt":{"rendered":"

              Due to\u00a0an increase\u00a0in service providers in the cloud computing domain, there is a lot of options available for the organizations to choose a platform for storing their data, processing it, presenting\u00a0it,\u00a0and\u00a0using\u00a0advanced\u00a0technologies. Modern data platform is a\u00a0set of different tools that are\u00a0brought together to manage and use the data efficiently for organizations to take data driven […]<\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[249],"tags":[],"class_list":["post-13919","post","type-post","status-publish","format-standard","hentry","category-dataops"],"_links":{"self":[{"href":"https:\/\/ismiletechnologies.com\/en_us\/wp-json\/wp\/v2\/posts\/13919","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ismiletechnologies.com\/en_us\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ismiletechnologies.com\/en_us\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ismiletechnologies.com\/en_us\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/ismiletechnologies.com\/en_us\/wp-json\/wp\/v2\/comments?post=13919"}],"version-history":[{"count":2,"href":"https:\/\/ismiletechnologies.com\/en_us\/wp-json\/wp\/v2\/posts\/13919\/revisions"}],"predecessor-version":[{"id":36171,"href":"https:\/\/ismiletechnologies.com\/en_us\/wp-json\/wp\/v2\/posts\/13919\/revisions\/36171"}],"wp:attachment":[{"href":"https:\/\/ismiletechnologies.com\/en_us\/wp-json\/wp\/v2\/media?parent=13919"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ismiletechnologies.com\/en_us\/wp-json\/wp\/v2\/categories?post=13919"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ismiletechnologies.com\/en_us\/wp-json\/wp\/v2\/tags?post=13919"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}