DoubleCloud
DoubleCloud
  • 42
  • 46 801
DoubleCloud’s Data Transfer as a CliсkHouse native ELT tool
DoubleCloud’s Data Transfer is the best tool on the market for moving data to and from ClickHouse. With us, you can replicate data quickly, scale easily, and create snapshots. Our transfer service is built for scale, which is reflected in price and performance.
See our Data Transfer in action! You’ll learn:
- What makes our transfer service ClickHouse-native
- How to set up Data Transfer for ClickHouse using DoubleCloud
- What Data Transfer can bring to the table if you self-manage ClickHouse
The DoubleCloud team includes people who worked on creating the first managed ClickHouse. We’ve had our share of experience working with this exceptional database and have built tools that help you get the most out of it. And, as you’ve probably guessed, we like to talk about it.
🔍 DoubleCloud services mentioned in the video:
No-code ELT tool: Data Transfer 👉 double.cloud/services/doublecloud-transfer/?
Managed Service for ClickHouse® 👉 double.cloud/services/managed-clickhouse/?
Managed Service for Apache Kafka® 👉 double.cloud/services/managed-kafka/?
Data Visualization with AI-insights 👉 double.cloud/services/doublecloud-visualization/?
🔗 Website: double.cloud/?
📚 Related content:
Transferring data… Everything you’ll ever need to know ► double.cloud/blog/posts/2022/12/transferring-data-everything-you-ll-ever-need-to-know/?
ETL vs ELT: Choosing the right approach for your data integration needs ► double.cloud/blog/posts/2023/01/etl-or-elt-do-you-know-the-difference/?
ETL vs data pipelines: What are they and how do they work? ► double.cloud/blog/posts/2023/06/etl-vs-datapipelines/?
Chapters:
00:00 Introduction: Who are we?
02:31 What is ClickHouse and why is a good data base for modern analytics?
04:26 How to write data into ClickHouse efficiently
05:38 How to integrate data in real world?
6:15 Typical analytics use cases with ClickHouse
08:03 Real-time streaming analytics use case
9:33 Pros and cons of Data Transfer for real-time streaming analytics
11:26 Data Warehouse use case
16:50 Pros and cons of Data Transfer for Data Warehouse use case
18:00 DoubleCloud’s transfer as a ClickHouse native tool explained
19:15 Demo of DoubleCloud’s transfer service
29:44 Demo of DoubleCloud’s Data Visualization
33:00 Q&A: Questions answered
📱 Follow us on Social Media:
LinkedIn: www.linkedin.com/company/doublecloudplatform/
Facebook: GetDoubleCloud/
Twitter: getdoublecloud
Medium: medium.com/doublecloud-insights
Reddit: www.reddit.com/user/GetDoubleCloud/
🆓 Start building your data infrastructure today with our free trial: auth.double.cloud/s/signup
📥 Have a question or feedback? Drop us a comment below or reach out through double.cloud/#contact-us-form
Remember to subscribe for more DoubleCloud insights and access to big-data discussions with experts. 💡
มุมมอง: 31

วีดีโอ

Using ClickHouse for real time analytics
มุมมอง 36821 วันที่ผ่านมา
Welcome to our in-depth session on setting up real-time analytics with ClickHouse, one of the world’s fastest databases! In this comprehensive webinar, you'll learn about the powerful features that make ClickHouse a top choice for real-time analytics. Follow our detailed guide to set up and configure ClickHouse clusters with ease, and discover best practices to ensure your ClickHouse setup runs...
Build your analytics solutions with DoubleCloud in 10 minutes
มุมมอง 3622 หลายเดือนก่อน
Watch our step-by-step demo to learn how to set up a managed ClickHouse or Kafka cluster in 10 minutes, utilizing the web UI or Terraform. Build data analytics infrastructure in a few clicks with DoubleCloud and enjoy exceptional performance for an affordable price. 🆓 Start today with our free trial: auth.double.cloud/s/signup 🔗 Website: double.cloud/ 🔍 DoubleCloud services mentioned in the vid...
5 tips to boost real-time analytics performance with Managed Kafka
มุมมอง 583 หลายเดือนก่อน
Attention, data engineers! Are you finding Apache Kafka a bit of a tough nut to crack? Every data engineer encounters this challenge at some point. Fear not, because we’ve got you covered! Join us for a webinar where our tech lead will spill the beans on how to improve your Managed Kafka performance. Say goodbye to sluggish real-time analytics and hello to lightning-fast data streams! 00:00 Int...
Managed open source vs proprietary data stack
มุมมอง 2314 หลายเดือนก่อน
Managed open source vs proprietary data stack
Why use Terraform to stream data with Apache Kafka?
มุมมอง 2414 หลายเดือนก่อน
Join us for an insightful webinar where DoubleCloud experts guide you through the seamless setup of a Change Data Capture (CDC) streaming environment using Apache Kafka, all streamlined with the power of Terraform. Gain practical knowledge as we walk you through the step-by-step process of setting up and optimizing your data streaming pipeline, ensuring efficient and effective data replication....
ClickHouse Bootcamp: Strategies for hybrid storage and data insertion
มุมมอง 4.9K4 หลายเดือนก่อน
In this informative webinar, we delve into the technical aspects of ClickHouse and explore topics around hybrid storage and more. Everyone knows that ClickHouse is an extremely fast and cost-effective database. As your data grows or data application requirements evolve, you may face some common challenges. How do you keep your costs in-check without sacrificing performance? What are the best wa...
The right integration method to minimize delivery lag: Kafka and ClickHouse for real-time analytics
มุมมอง 4.9K4 หลายเดือนก่อน
Choosing the right integration method to minimize delivery lag. ClickHouse and Kafka, scalable and open-source, form a powerful duo for building cost-effective, real-time analytics systems, suitable for log analysis, clickstream analytics, financial transactions, and more. The integration method of this combination is an important consideration that affects ingestion performance, resource distr...
How to deploy real-time analytics in 10 minutes with DoubleCloud's Managed Kafka
มุมมอง 1224 หลายเดือนก่อน
In this video we discuss the features, challenges, and benefits of using Kafka in different deployment scenarios, comparing self-managed installations with proprietary solutions offered by vendors like DoubleCloud. It covers various aspects such as scalability, monitoring, infrastructure management, Kafka Connect capabilities, and integration with other tools like ClickHouse for real-time analy...
How to elevate your workflow automation with managed Apache Airflow
มุมมอง 725 หลายเดือนก่อน
Tired of juggling complex data pipelines and task orchestration? In this video, we unveil the power of DoubleCloud Managed Apache Airflow - our new tool for efficient workflow orchestration. Don’t miss out on this incredible opportunity to transform your workflows and boost your productivity. In this webinar, we cover a range of topics: 00:00 Introduction 01:34 Data orchestration 02:58 Introduc...
How to build fast and reliable end-user analytics: Pitfalls and how-tos for CTOs
มุมมอง 1605 หลายเดือนก่อน
In this video, we delve into the challenges businesses face throughout the data lifecycle and explore techniques for extracting value that satisfies CEOs, teams, and peers. Discover effective approaches to customer-facing analytics and cost-saving methods using open-source solutions. Gain practical insights from real customer experiences, including achieved results and essential building blocks...
How to build end-to-end analytics in 1 week? Food delivery tech leader experience
มุมมอง 3065 หลายเดือนก่อน
In this recorded webinar, explore Yango Tech's success story in the African food delivery market with their Chief Data Officer (CDO). We investigate: Is providing partners with sub-second analytics the key to growth? Uncover their rapid success through an innovative partner model, providing cutting-edge technologies and empowering partners with data analytics tools to drive sales. 00:00 Introdu...
A ClickHouse Bootcamp in 30 minutes: Tips & tricks to optimize your data from industry experts
มุมมอง 4075 หลายเดือนก่อน
In this video, we're excited to share an exclusive Bootcamp on enhancing your analytics with ClickHouse. ClickHouse, a high-performance analytics database, has the potential to revolutionize your insights and keep you ahead of the competition. Our hosts have more than 7 years of hands-on experience with ClickHouse, and in this video, they share best practices, including performance optimization...
Real-time analytics for IoT & robotics: How to enhance decision-making with faster data processing?
มุมมอง 2815 หลายเดือนก่อน
In this recorded webinar, discover strategies for real-time monitoring and optimization in robotics and IoT companies. Learn to maximize efficiency and cut maintenance costs with insights from our experts. Explore real-world customer use cases and technical implementation details using tools like Grafana and DoubleCloud Managed Clickhouse. Gain valuable insights into runtime analytics, robot ma...
PostgreSQL vs ClickHouse: Query performance test
มุมมอง 3885 หลายเดือนก่อน
In this video, we test the speed difference between PostgreSQL and ClickHouse in our performance test! Watch as we run a basic aggregation query in PostgreSQL and then in ClickHouse, followed by a more complex index aggregation query. The numbers speak for themselves: 8.1 seconds for PostgreSQL versus 0.4 seconds for ClickHouse in the basic query, and 25 seconds for PostgreSQL versus 1.8 second...
How to boost your PostgreSQL performance with ClickHouse for instant analytics
มุมมอง 5015 หลายเดือนก่อน
How to boost your PostgreSQL performance with ClickHouse for instant analytics
ClickHouse vs. Elasticsearch: Which data storage and analytics solution suits your requirements?
มุมมอง 3645 หลายเดือนก่อน
ClickHouse vs. Elasticsearch: Which data storage and analytics solution suits your requirements?
The ultimate guide to data analytics for EdTech product innovation
มุมมอง 1185 หลายเดือนก่อน
The ultimate guide to data analytics for EdTech product innovation
How to build robust data flows: ETL and data pipelines best practices
มุมมอง 2915 หลายเดือนก่อน
How to build robust data flows: ETL and data pipelines best practices
Apache Kafka vs Apache Spark: Choosing the right streaming platform for your business
มุมมอง 1285 หลายเดือนก่อน
Apache Kafka vs Apache Spark: Choosing the right streaming platform for your business
How to maximize real-time data value? Dive into streaming analytics exploration
มุมมอง 1375 หลายเดือนก่อน
How to maximize real-time data value? Dive into streaming analytics exploration
DoubleCloud's Visualization: Gain insights faster with AI-Insights
มุมมอง 11Kปีที่แล้ว
DoubleCloud's Visualization: Gain insights faster with AI-Insights
Insightful visualization: Unlocking the potential of your data
มุมมอง 694ปีที่แล้ว
Insightful visualization: Unlocking the potential of your data
Why you should use the DoubleCloud Platform
มุมมอง 240ปีที่แล้ว
Why you should use the DoubleCloud Platform
How to create a Kafka cluster on DoubleCloud
มุมมอง 128ปีที่แล้ว
How to create a Kafka cluster on DoubleCloud
How to connect to your ClickHouse cluster in DoubleCloud?
มุมมอง 69ปีที่แล้ว
How to connect to your ClickHouse cluster in DoubleCloud?
How to change user permissions in DoubleCloud
มุมมอง 38ปีที่แล้ว
How to change user permissions in DoubleCloud
How to add users to your DoubleCloud's managed instance
มุมมอง 51ปีที่แล้ว
How to add users to your DoubleCloud's managed instance
How to decrease the cost of your observability solution by building it in open-source
มุมมอง 441ปีที่แล้ว
How to decrease the cost of your observability solution by building it in open-source
How to use DoubleCloud's Data Visualization tool
มุมมอง 198ปีที่แล้ว
How to use DoubleCloud's Data Visualization tool

ความคิดเห็น

  • @kenroygeorge
    @kenroygeorge 12 วันที่ผ่านมา

    Hello Andrei, thank you for creating this amazing walkthrough. What is the main reason for adding Kafka to this setup instead of writing and reading directly from ClickHouse? Is it a security or performance issue?

    • @laskoviymishka
      @laskoviymishka 9 วันที่ผ่านมา

      There are two main reasons: 1) performance of ingestion, clickhouse hate small inserts, so before writing into it you must buffer you data somehow. Rule of thumb - you should write into clickhouse 1-2 times per second in big batches 2) decoupling of consumer / producer. If you add kafka into a setup you will be able to read this data multiple times, and to add new destination you will need to touch your producer code. In general kafka here - is a good thing, but indeed optional. For many cases writing directly to clickhouse is totally fine, but code that writes to a clickhouse properly may be abscure.

    • @kenroygeorge
      @kenroygeorge 9 วันที่ผ่านมา

      @@laskoviymishka Thank you very much for the detailed explanation. I will dig into Kafka some more to understand it. From what I understand though, wouldn't the messages be coming off the queue one by one, hence still being small commits when ClickHouse is consuming messages?

    • @laskoviymishka
      @laskoviymishka 9 วันที่ผ่านมา

      @@kenroygeorge Usually you glue Kafka and clickhouse together via some tool (like in this video via double-cloud transfer). Those tools are doing such buffers seamlessly to user, but it still happens on the backend side.

  • @superboyke1065
    @superboyke1065 หลายเดือนก่อน

    😢😮😮😅

  • @RT-.
    @RT-. 2 หลายเดือนก่อน

    But how big are the datasets?

    • @doublecloud249
      @doublecloud249 2 หลายเดือนก่อน

      The datasets used in our performance test were approximately 100GB in size. However, it's important to note that even with smaller tables, such as 10GB, PostgreSQL exhibited performance slowdowns.

  • @aibunny_
    @aibunny_ 4 หลายเดือนก่อน

    Loving this I saw parade db also another open source olap db is almost as good as clickhouse

  • @adamsaffron6721
    @adamsaffron6721 4 หลายเดือนก่อน

    What do you guys think of RisingWave?

  • @oscarpowers468
    @oscarpowers468 4 หลายเดือนก่อน

    "promo sm" 🏃

  • @MohammedMisselmany
    @MohammedMisselmany 5 หลายเดือนก่อน

    AWESOME!

  • @user-rx1ty4qw4e
    @user-rx1ty4qw4e ปีที่แล้ว

    Really liked the short video and to the point format. Keep it up!

  • @xdxd9853
    @xdxd9853 ปีที่แล้ว

    0poo

  • @miguelhurst559
    @miguelhurst559 2 ปีที่แล้ว

    Come on guys, we all know the best is there on this channel

  • @kayleighosborne5720
    @kayleighosborne5720 2 ปีที่แล้ว

    Pretty good....well done

  • @wilberthintz381
    @wilberthintz381 2 ปีที่แล้ว

    Respect! :) thats so amazing, wow!

  • @suzannebernard8894
    @suzannebernard8894 2 ปีที่แล้ว

    Soo is grateful for this video!!!! Keep up the good work!

  • @robertblevins242
    @robertblevins242 2 ปีที่แล้ว

    This worked PERFECTLY