- 13
- 12 057
Quickwit
United States
เข้าร่วมเมื่อ 4 มี.ค. 2022
Search more with less. The open source search engine on object storage with subsecond latency.
A Deep Dive into the Rust crate Tower
Tower is a crate of “modular and reusable components for building robust networking clients and servers” widely used in prominent projects within the Rust ecosystem, including Hyper*, Axum, or Tonic.
This ubiquity means Rust developers invariably engage with the Service trait, Tower's core abstraction while working with these libraries. Despite its conciseness, reading and understanding code that abstracts over the Service trait proves challenging because of the heavy use of generics and intricacies of implementing non-boxed futures.
This complexity raises the learning curve for beginner and intermediate Rust developers to become proficient in their favorite frameworks. In this talk, we will demystify the Service trait, explore practical examples from real-world codebases, and construct increasingly complex Tower layers from scratch.
*Hyper actually relies on its own very similar hyper::Service trait.
Author: Adrien Guillo, co-founder of Quickwit
This ubiquity means Rust developers invariably engage with the Service trait, Tower's core abstraction while working with these libraries. Despite its conciseness, reading and understanding code that abstracts over the Service trait proves challenging because of the heavy use of generics and intricacies of implementing non-boxed futures.
This complexity raises the learning curve for beginner and intermediate Rust developers to become proficient in their favorite frameworks. In this talk, we will demystify the Service trait, explore practical examples from real-world codebases, and construct increasingly complex Tower layers from scratch.
*Hyper actually relies on its own very similar hyper::Service trait.
Author: Adrien Guillo, co-founder of Quickwit
มุมมอง: 2 206
วีดีโอ
Modern application observability with Grafana and Quickwit
มุมมอง 9579 หลายเดือนก่อน
In the world of modern distributed applications, detailed performance monitoring is critical. New distributed tracing tools like Jaeger or, more recently, Grafana Tempo help drill down to individual transactions and understand application-level performance. However, with millions of users or request IDs, capturing metrics for high cardinality fields is a real challenge. In this talk, we will in...
Building an actor library for Quickwit's indexing pipeline
มุมมอง 377ปีที่แล้ว
Talk recorded at FOSDEM 2023: fosdem.org/ Related blog post quickwit.io/blog/quickwit-actor-framework
Search and analytics on 5.6 billions GitHub events ~ 21TB JSON ~ on AWS S3
มุมมอง 587ปีที่แล้ว
Quickwit reposutory: github.com/quickwit-oss/quickwit Dataset: GitHub archive dataset with 5.6 billion event ~ 21TB of JSON. Storage: all data are stored on AWS S3. Compute: 3 searchers with 8vCPU each. Query 1: LGTM review given on Quickwit PR ```query=repo.name:quickwit AND payload.review.body:LGTM``` Response time: less than 2 seconds Query 2: top rust contributors ``` { "query": "org.login:...
How we built a DISTRIBUTED search engine in Rust
มุมมอง 1.9Kปีที่แล้ว
In a data-driven world, having to search through terabytes of logs is common. This simple task can turn out to be daunting and very expensive. This talk will present how to implement such an engine with an efficient architecture strongly inspired by Snowflake which separated compute and storage. 🦀 Thank you RustLab for organizing the event! Check out more of their events on their TH-cam channel...
Cloud-Native Search Engine for Log Management and Analytics, Cloud-Native Meetup | Adrien Guillo
มุมมอง 6252 ปีที่แล้ว
In this talk, Adrien Guillo, Co-Founder of Quickwit, will cover: - Cloud-nativeness limitations of traditional search engines - Quickwit’s novel approach and architecture - Production use cases of Quickwit - Demo ☁️ Thank you Cloud-Native New York for organizing the event! Check out their content: - Their TH-cam Channel: th-cam.com/channels/5xpPLzD2_sZpg640_phYqw.html - Their Meetup: www.meetup...
Cost-Efficient Rust in Practice, Rust Tokyo Meetup | Paul Masurel, Co-Founder
มุมมอง 1.2K2 ปีที่แล้ว
How can Tantivy index the entire English Wikipedia on a single desktop in just a few minutes? How does Quickwit reach a throughput of 1GB/s on a k8s cluster? Paul Masurel (Co-Founder & CEO of Quickwit) will tell about building an open-source, cloud-native search engine from the ground up in Rust. Watch to learn about the features, design, internals, and challenges of developing Tantivy and Quic...
Building a Highly Efficient Search Engine for Logs In Rust | Francois, the Co-Founder of Quickwit
มุมมอง 1K2 ปีที่แล้ว
In today’s modern world, any systems/servers/applications produce an increasing tremendous amount of logs and you often have to choose between costly SaaS services or on-premise solutions that are hard to manage at a large scale. That’s why we build Quickwit from the ground up to make it cloud-native. In this talk, we will see how we re-designed indexing and search to truly decouple storage and...
LNX: Using Tantivy to Build One of the Fastest Search Engines Around | Harrison, Software Engineer
มุมมอง 1.2K2 ปีที่แล้ว
LNX aims to be one of the most performant and cost-effective search engine alternatives to Elasticsearch and Algolia for typo-tolerant search. It offers a wide range of features thanks to the ecosystem it stands on, including but not limited to: complex query parser, typo-tolerant fuzzy queries, typo-tolerant fast-fuzzy queries (pre-computed spell correction), more-like-this queries. It thrives...
ChitChat: Cluster Membership with Failure Detection in Rust | Evance, Software Engineer
มุมมอง 1782 ปีที่แล้ว
Cluster membership management is one of the critical aspects of a distributed system. To put it simply, nothing can work if nodes don’t know about each other and what service or resource each can offer. In this talk, first, we briefly describe Quickwit’s previous cluster management implementation. Then deeply explore the new implementation based on the scuttlebutt algorithm while highlighting o...
QST | Cluster Membership Algorithm with Scuttlebutt
มุมมอง 4872 ปีที่แล้ว
The reason why we are moving away from SWIM protocol - Gossip with Scuttlebutt. The Quickwit's cluster membership algorithm implemented in Rust Programming Language. Subscribe for more developer content like this : th-cam.com/channels/vZVuRm2FiDq1_ul0mY85wA.html Benefits of our membership cluster algo "Scuttlebutt" include, but are not limited to : - Easier to understand and implement correctly...
Hello, I've a Quickwit ec2 instance in account A, could I configure a Kinesis source from account B? Thanks for your time
This is amazing.
Thanks so much for this amazing video. Thank you Thank you !!!
Can i have a video of how to use and create indexes within a docker container
Let's do this!!!!
@@quickwit8103 Have done this thanks ... Yet i need clarification on indexing the content and displaying results .. if possible make a complete tutorial
The service trait here looks reminds me so much of Finagle.
is that grafana dashboard anywhere on github?
If I could, I would give it a thumbs up 3 times. The question and answer session answered all my questions!
I have recently started doing rust and this was indeed a great intro for tower, Thanks for this <3
Lmao love this so much
not me thinking of the game rust
The clown masks. Dimwits.
This is the most exciting development in the search engine land.
ᴘʀᴏᴍᴏsᴍ
👋Let us know if you have any questions about the topics/Quickwit and we will reply right here!
👀Let us know all your questions in the comments! The Quickwit team will be happy to answer.
👀Let us know all your questions in the comments! The Quickwit team will be happy to answer.
👀Let us know all your questions in the comments! The Quickwit team will be happy to answer.
Feel free to ask your questions here and we will...ingest answers into them!
Can you post the latest "Stream Ingestion with Apache Kafka & Amazon Kinesis"?
Hi Marko, the event was postponed to the 8th of June because of an outage of restream.io the day of the event. You can subscribe here: quickwit.io/events/stream-ingestion-with-kafka-and-kinesis/
Good work
🚀🚀🚀