Разработка • dev.to • 31 мая 2024 г. 17:31

Working with geospatial data is really fun! We use this type of data all over the place in data science, and data warehouses have gotten really good at working with it (I'm looking at you, Snowflake!). However, as you get started down the path of us...... читать далее

Разработка • dev.to • 29 мая 2024 г. 18:45

At the time of writing this, I'm using python Pybit v5 library for scrapping Bybit kline data. Following python codes work for 1min kline scrapping, do modify the code accordingly based the interval you wish to scrap. import pandas as pd from date...... читать далее

Разработка • dev.to • 28 мая 2024 г. 12:34

Machine learning is the use of algorithms that can learn from data over time and therefore can detect and learn patterns from the data. Machine learning models are divided into Supervised, Unsupervised, and Reinforcement learning. The commonly used m...... читать далее

Разработка • realpython.com • 27 мая 2024 г. 14:00

In this tutorial, you'll learn how to create pivot tables using pandas. You'll explore the key features of DataFrame's pivot_table() method and practice using them to aggregate your data in different ways.... читать далее

Разработка • dev.to • 26 мая 2024 г. 14:08

Data cleaning is a crucial step in any data analysis or machine learning project. It involves identifying and correcting errors, handling missing values, and ensuring the data is in a suitable format for analysis. In this blog, we will explore data c...... читать далее

Разработка • dev.to • 25 мая 2024 г. 20:47

In order to understand Spark let's remember what was the scenario before its creation. A couple of years ago computers became faster every year through processor speed increases. This trend in hardware stopped around 2005 due to hard limits in heat d...... читать далее

Разработка • dev.to • 23 мая 2024 г. 0:31

Learn about the different feature types that can be part of a dataset. In the context of data analysis using pandas DataFrames in Python, understanding the difference between quantitative and categorical characteristics is crucial. Let's break down...... читать далее

Разработка • Facebook Engineering • 22 мая 2024 г. 17:15

In recent years, Meta’s data management systems have evolved into a composable architecture that creates interoperability, promotes reusability, and improves engineering efficiency.  We’re sharing how we’ve achieved this, in part, by leveraging Velox...... читать далее

Разработка • dev.to • 22 мая 2024 г. 5:00

Purpose of the notebook Early on with this blog I wanted a way to track its usage, just to see if anyone was reading it, and it needed to be GDPR compliant. Enter Plausible Analytics, it was GDPR compliant without needing the cookie or consent bann...... читать далее

Разработка • dev.to • 21 мая 2024 г. 21:44

In this post we show a quick step-by-step guide to collecting data from "my" publications on DEV (dev.to) using its beta API. We use Python 3.9+ libraries (Requests, Json and Pandas) to make requests to DEV API endpoints, then pass them to the DataFr...... читать далее

Разработка • dev.to • 21 мая 2024 г. 5:45

We have a database of a fictional telco company providing home phone and internet services to customers in California. We want to build an AI system capable of analizing the data and answering users' questions about customers, services, and billing...... читать далее

Разработка • dev.to • 21 мая 2024 г. 3:14

Pythonic reverse ETL is here Why Python is the right approach for doing Reverse ETL Reverse ETL is generally about putting data into a business application. This data would often come from a SQL database used as a middle layer for data i...... читать далее

Разработка • dev.to • 21 мая 2024 г. 1:06

If you are a data enthusiast, you must have heard about the powerful library in Python called Pandas. Pandas is a versatile tool that is widely used by data scientists, analysts, and programmers for data manipulation, analysis, and visualization. In...... читать далее

Разработка • dev.to • 20 мая 2024 г. 19:09

As a passionate developer with many years of experience, I've witnessed the evolution of technology and its impact on the way we handle data. While traditional methods have served us well, the advent of Generative AI (GenAI) presents a transformative...... читать далее

Разработка • dev.to • 19 мая 2024 г. 21:40

Introduction Welcome to this comprehensive guide on using Python for data analysis! Whether you're a beginner or an experienced programmer, this post will provide valuable insights into harnessing Python's power for your data projects. We'll cover e...... читать далее

Разработка • dev.to • 19 мая 2024 г. 13:16

Have you wondered how to write Hudi tables (Scala) in AWS Glue? Look no further. Pre-requisites Create a Glue Database called hudi_db from the Databases under Data Catalog menu in the Glue Console Let's pick the Apache Hudi Spark QuickS...... читать далее

Разработка • dev.to • 19 мая 2024 г. 11:39

Neste post mostramos um passo a passo rápido para coletar dados de "minhas" publicações no DEV (dev.to) utilizando sua API versão beta. Usamos bibliotecas do Python 3.9+ (Requests, Json e Pandas) para realizar requisições a endpoints da API DEV, entã...... читать далее

Разработка • dev.to • 18 мая 2024 г. 16:57

Reading and loading data from a cosmos db data in a dataframe but there is a field which is empty like {} which i want to replace or remove because while writing the same data in parquet file, it is giving error like datasource dont support empty s...... читать далее

Разработка • dev.to • 18 мая 2024 г. 12:15

How to deserialize a DataFrame dict into faust Record? May 18 '24 Comments: 1 Answers: 0 0 I'm...... читать далее

Разработка • dev.to • 17 мая 2024 г. 12:30

In the world of software development, encountering bugs is an inevitable part of the process. Some bugs are predictable and can be tackled with routine testing, while others surface unexpectedly, causing delays and frustration. Interestingly, there a...... читать далее