DevOps • DZone DevOps • 24 июля 2023 г. 13:17
Businesses in the modern, data-driven economy significantly rely on data to make wise decisions. A data warehouse is an essential part of data architecture because it offers a centralized location for storing, managing, and analyzing massive amounts...... читать далее
Безопасность • DZone Security • 16 июля 2023 г. 20:03
Data Platform Evolution Initially, data warehouses served as first-generation platforms primarily focused on processing structured data. However, as the demand for analyzing large volumes of semi-structured and unstructured data grew, secon...... читать далее
DevOps • DZone DevOps • 12 июля 2023 г. 15:39
Maybe you’ve noticed the world has dumped the internet, mobile, social, cloud, and even crypto in favor of an obsession with generative AI. But is there more to generative AI than a fancy demo on Twitter? And how will it impact data? ... читать далее
Разработка • DZone Web Dev • 11 июля 2023 г. 15:20
Snowpipe Streaming Snowflake introduced Snowpipe Streaming and is set to be a key technology for near-real-time streaming workloads. Data ingestion is, however, only one part of the story.... читать далее
Разработка • AWS News Blog • 3 июля 2023 г. 18:20
In last week’s AWS Week in Review post, Danilo mentioned that it’s summer in London. Well, I’m based in Singapore, and it’s mostly summer here. But, June is a special month here as it marks the start of durian season. Starting next week, I’ll be trav...... читать далее
Amazon OpenSearch Service Amazon SageMaker Data Wrangler Announcements AWS Cloud Development Kit News Uncategorized Week in ReviewРазработка • dev.to • 30 июня 2023 г. 20:28
In this article, we will talk about data engineers, what they manage, what they develop, and other activities they perform. We will also explore what a BI architecture is and what role data engineers play in this vast world of data (which is still re...... читать далее
dataengineering datascience beginners architectureРазработка • dev.to • 26 июня 2023 г. 1:10
26-June-2023 FLiPN-FLaNK Stack Weekly Tim Spann @PaaSDev My friend wrote an awesome new book on streaming, I highly recommend picking up a copy! https://leanpub.com/streamprocessingwithapacheflink/c/ucQ5dLcZYAo2 Join me in pe...... читать далее
apacheflink opensource apachenifi apachekafkaDevOps • DZone DevOps • 20 июня 2023 г. 19:14
In every organization, there may be multiple source systems for various needs. Depending on how big the organization is, the source systems may vary from one to more than 1,000. Organizations often centralize their data in one place, integrate it, an...... читать далее
Разработка • dev.to • 17 июня 2023 г. 14:02
In SQL and data warehousing, dimension and fact tables are two types of tables used in a star schema or snowflake schema to model and organize data. They serve different purposes and play key roles in data analysis and reporting. Here are definitions...... читать далее
datawarehousing sql analytics datamodellingРазработка • dev.to • 14 июня 2023 г. 21:58
Guest blog by Shashank Mishra, Data Engineer @ Expedia TLDR Amazon Redshift is a powerful, scalable data warehousing service within the AWS ecosystem. It excels in handling large datasets with its columnar storage, parallel query executio...... читать далее
aws redshift datawarehousing dataengineeringDevOps • DZone DevOps • 12 июня 2023 г. 15:57
In the previous article, we understood how to Optimally Use Snowflake Warehouse and Tables. So let’s continue this blog series, where we will now focus on improving the performance in Snowflake and its associated costs.... читать далее
Разработка • dev.to • 8 июня 2023 г. 14:24
In today's data-driven world, efficient data management and processing are crucial for businesses to gain valuable insights and make informed decisions. This is where data build tools come into play. In this guide, we will explore the concept of da...... читать далее
data datascience python sqlРазработка • Cloudera Blog • 5 июня 2023 г. 20:13
Results from the “Best in Flow” data pipeline development contest with Cloudera DataFlow in the public cloud The post Aaand the New NiFi Champion is… appeared first on Cloudera Blog.... читать далее
TechnicalDevOps • DZone DevOps • 2 июня 2023 г. 11:40
In a recent survey by Great Expectations, 91% of respondents revealed that data quality issues had some level of impact on their organization. It highlights the critical importance of data quality in data engineering pipelines. Organizations can avoi...... читать далее
DevOps • DZone DevOps • 31 мая 2023 г. 15:05
In this detailed and personal account, the author shared his journey of building and evolving data pipelines in the rapidly transforming streaming media industry. Drawing from his extensive experience, the author highlights the fundamental role data...... читать далее
Разработка • dev.to • 30 мая 2023 г. 5:56
In the realm of software development and data management, the efficiency of data retrieval plays a crucial role in the overall performance of an application. One effective way to enhance this efficiency is by implementing a cache. A cache is a high-s...... читать далее
postgres cache api systemsDevOps • DZone DevOps • 29 мая 2023 г. 12:46
Data freshness, sometimes referred to as data timeliness, is the frequency with which data is updated for consumption. It is an important dimension of data quality because recently refreshed data is more accurate and, thus, more valuable. Since it is...... читать далее
Разработка • dev.to • 27 мая 2023 г. 14:00
Machine Learning Operations, or MLOps, is the missing bridge between machine learning, data science, and data engineering. It has emerged as the link that unifies these functions more seamlessly than ever before. So what is MLOps, and why does it mat...... читать далее
mlops devops dataoperations continuousdeliveryРазработка • The Cloudflare Blog • 22 мая 2023 г. 13:00
Developer Week 2023 is officially a wrap. Let’s take a look at everything we shipped last week... читать далее
Developers Developer Week Product NewsDevOps • DZone DevOps • 19 мая 2023 г. 19:35
If you've been struggling to find a way to seamlessly connect Snowflake and BigQuery, then this article is a great resource for you. You already know that in today's data-driven world, integrating your data sources is essential for making informed de...... читать далее
