DataStax on foojay.io - Friends Of OpenJDK

Foojay Podcast #56: Vectors in Java Code, Databases, and LLMs

Mon, 08 Jul 2024 07:33:28 +0000

In this Foojay podcast, we enter the world of mathematics by discussing Vectors and how they are crucial for AI and machine learning.

As ChatGPT explains: “A Vector is a mathematical structure that holds numerical values. Vectors are fundamental to the field of Artificial Intelligence, as they allow mathematical operations to be performed efficiently and form the basis of many machine learning algorithms.”

OK, but… how are these vectors crucial for the whole Artificial Intelligence evolution?

Indexing all of Wikipedia, on a laptop

Wed, 29 May 2024 15:54:32 +0000

In November, Cohere released a dataset containing all of Wikipedia, chunked and embedded to vectors with their multilingual-v3 model.

Computing this many embeddings yourself would cost in the neighborhood of $5000, so the public release of this dataset makes creating a semantic, vector-based index of Wikipedia practical for an individual for the first time.

Here’s what we’re building:

You can try searching the completed index on a public demo instance here.

Unified Event-Driven Architecture for the Cloud Native Enterprise

Thu, 06 Oct 2022 19:58:06 +0000

CTOs and enterprise architects have long recognized the importance of event-driven architectures (EDA). While once considered purely a technology concern, the foresight of organizations that have invested in EDA has become readily apparent as the world has shifted around us. In the past decade, we’ve witnessed changes in nearly every aspect of our technological worlds, and the vast majority of those have been affected in some way by a move toward event-driven, real-time processing.

Reclaiming Persistent Volumes in Kubernetes

Wed, 28 Sep 2022 09:55:19 +0000

Kubernetes is a widely used open-source container management platform for running stateless, containerized applications at scale. In recent years, Kubernetes has been extended to also support stateful workloads, including databases and key-value stores.

There are three important API resources when it comes to managing stateful applications in Kubernetes:

STSs schedule stateful pods, which can claim PVs through PVCs and mount them as volumes. Once a PV is claimed by an STS replica, Kubernetes will make sure that the volume stays with the replica, even if the pod gets rescheduled.

How the world caught up with Apache Cassandra

Thu, 15 Sep 2022 16:50:50 +0000

The O’Reilly book, Cassandra: The Definitive Guide, features a quote from Ray Kurzweil, the noted inventor and futurist:

“An invention has to make sense in the world in which it is finished, not the world in which it is started.”

This quote has a prophetic ring to it, especially considering my co-author Eben Hewitt included it in the 2010 first edition of this book we wrote, back when Apache Cassandra, the open-source, distributed, and highly scalable NoSQL database, was just on its 0.7 release.

Baeldung Series Part 2: Build a Dashboard With Cassandra, Astra and CQL – Mapping Event Data

Tue, 13 Sep 2022 17:46:02 +0000

1. Introduction

In our previous article, we looked at augmenting our dashboard to store and display individual events from the Avengers using DataStax Astra, a serverless DBaaS powered by Apache Cassandra using Stargate to offer additional APIs for working with it.

In this article, we will be making use of the exact same data in a different way. We are going to allow the user to select which of the Avengers to display, the time period of interest, and then display these events on an interactive map. Unlike in the previous article, this will allow the user to see the data interacting with each other in both geography and time.

Multi-cluster Cassandra with Google Kubernetes Engine (Pt. 2)

Thu, 08 Sep 2022 17:20:08 +0000

This is the second in a series of posts examining patterns for using K8ssandra to create Cassandra clusters with different deployment topologies.

In the first article in this series, we looked at how you could create a Cassandra cluster with two datacenters in a single cloud region, using separate Kubernetes namespaces in order to isolate workloads. For example, you might want to create a secondary Cassandra datacenter to isolate a read-heavy analytics workload from the datacenter supporting your main application.

Running your Database on OpenShift and CodeReady Containers

Tue, 06 Sep 2022 09:43:57 +0000

Let’s take an introductory run-through of setting up your database on OpenShift, using your own hardware and RedHat’s CodeReady Containers.

CodeReady Containers is a great way to run OpenShift K8s locally, ideal for development and testing. The steps in this blog post will require a machine, laptop or desktop, of decent capability; preferably quad CPUs and 16GB+ RAM.

Download and Install RedHat’s CodeReady Containers

Download and install RedHat’s CodeReady Containers as described in Red Hat OpenShift 4 on your laptop: Introducing Red Hat CodeReady Containers

Deploy a Multi-Datacenter Cassandra Cluster in Kubernetes

Wed, 10 Aug 2022 08:27:59 +0000

The Get Started examples on the K8ssandra site are primarily concerned with spinning up a single Apache Cassandra datacenter in a single Kubernetes cluster.

However, there are many situations that can benefit from other deployment options.

In this series of articles, we’ll examine different deployment patterns and show how to implement them using K8ssandra.

Flexible topologies with Cassandra

From its earliest days, Cassandra has included the ability to assign nodes to datacenters and racks. A rack was originally conceived as mapping to a single rack of servers connected to shared resources, like power, network, and cooling.

Five Data Models for IoT: Managing the Latest IoT Events Based on a State in Apache Cassandra

Thu, 28 Jul 2022 17:36:28 +0000

Apache Cassandra is a rock-solid choice for managing IoT and time series data at scale. The most popular use case of storing, querying and analyzing time series generated by IoT devices in Cassandra is well-understood and documented. In general, a time series is stored and queried based on its source IoT device. However, there exists another class of IoT applications that require quick access to the most recent data generated by a collection of IoT devices based on a known state. The question that such applications need to answer is: Which IoT devices or sensors are currently reporting a specific state? In this blog post, we focus on this question and provide five possible data modeling solutions to efficiently answer it in Cassandra.

NoSQL Use Cases: When to Use a Non-Relational Database

Thu, 28 Jul 2022 07:32:07 +0000

For decades, many companies have relied on relational databases to store, protect, and access their data.

SQL databases, in particular, worked well for a long time and still do for many use cases. But, today, there is a wide range of situations where SQL databases can no longer satisfy the needs of modern enterprises, especially those that have made the move to the cloud.

Increasingly, these companies are turning to NoSQL databases to meet their goals.

Kubernetes and the Data Centre: Is Full Scale Migration Possible?

Thu, 21 Jul 2022 13:48:13 +0000

Data centres used to be exactly that - the centres of our data worlds, where all information, files and records would be kept and controlled.

Today, that is no longer the case. More companies are moving to cloud services, whether this is dialling down on their traditional on-premises environments or never setting them up in the first place.

This is not to say that data centres will be wholly abandoned for cloud, but the mix will continue to evolve and be very different to today. Gartner predicts that by 2025, 85% of infrastructure strategies will take a more hybrid approach, integrating on-premises, colocation, cloud and edge delivery options, compared with 20% in 2020.

Why a Cloud-Native Database Must Run on K8s

Tue, 19 Jul 2022 17:44:09 +0000

We’ve been talking about migrating workloads to the cloud for a long time, but a look at the application portfolios of many IT organizations demonstrates that there’s still a lot of work to be done. In many cases, challenges with persisting and moving data in clouds continue to be the key limiting factor slowing cloud adoption, despite the fact that databases in the cloud have been available for years.

For this reason, there has been a surge of recent interest in data infrastructure that is designed to take maximum advantage of the benefits that cloud computing provides. A cloud-native database is one that achieves the goals of scalability, elasticity, resiliency, observability and automation; the K8ssandra project is a great example. It packages Apache Cassandra and supporting tools into a production-ready Kubernetes deployment.

The End of the Beginning for Apache Cassandra

Tue, 12 Jul 2022 17:43:39 +0000

Image: Pixabay

Editor’s note: This story originally ran on July 27, 2021, the day that Apache Cassandra 4.0 was released.

Today is a big day for those of us in the Apache Cassandra community. After a long uphill climb, Apache Cassandra 4.0 has finally shipped. I say finally, because it has at times seemed like an elusive goal. I’ve been involved in the Cassandra project for almost 10 years now and I have seen a lot of ups and downs.

Available Now - gRPC for Apache Cassandra

Thu, 07 Jul 2022 07:38:47 +0000

Build microservices easily with the NoSQL standard database

If you are like most developers, you are embracing applications built using microservices and a NoSQL database. There are many good reasons: faster time to market, lower total cost of ownership, better performance, less downtime, and easy scalability.

Creating cloud-native applications is hard because microservices are often written in different languages, database drivers have a maintenance burden, or familiar HTTP APIs lack the performance needed.

K8ssandra ramps up security features to match Kubernetes’ best practices

Thu, 23 Jun 2022 19:54:58 +0000

New security features are coming to the open-source data platform: K8ssandra. The goal? To align even more with the security best practices of Kubernetes. Here’s an introduction to the platform’s security mission and an update on current initiatives.

The security defaults applied by K8ssandra are about to get even more aligned withKubernetes’ security practices. In an upcoming release of K8ssandra, pod and container security configurations give users full customization capabilities and default values out of the box.

Data Modeling in Cassandra and Astra DB

Thu, 16 Jun 2022 19:22:05 +0000

What does it take to build an efficient and sound data model for Apache Cassandra® and DataStax Astra DB? Where would one start? Are there any data modeling rules to follow? Can it be done consistently time and time again? The answers to these and many other questions can be found in the Cassandra data modeling methodology.

In this post, we present a high-level overview of the data modeling methodology for Cassandra and Astra DB, and share over half a dozen complete data modeling examples from various real-life domains. We apply the methodology to create Cassandra and Astra DB data models for IoT, messaging data, digital library, investment portfolio, time series, shopping cart, and order management. We even provide our datasets and queries for you to try.

Tombstones and Ghost Data Don’t Have to Be Scary!

Thu, 09 Jun 2022 20:36:25 +0000

Working with tombstones can be one of the more frustrating aspects of Apache Cassandra®. Problems with tombstones can happen when a large number of deletes occur within a short period of time, resulting in slower queries and disk bloat. This article will discuss tombstones, including the purpose they serve, and strategies/methods on how to avoid them.

Tombstones in Apache Cassandra® are written markers that are created whenever a value is deleted. Their purpose is to indicate that any prior values have been deleted. When a read occurs, it pulls data from the sorted string table (SSTable) files on a node that may contain the data. All prior values for the requested key are returned, ordered by write-time.

Managing Distributed Applications in Kubernetes Using Cilium and Istio with Helm and Operator for Deployment

Tue, 07 Jun 2022 14:51:03 +0000

This post will show you the benefits of managing your distributed applications with Kubernetes in cross-cloud, multi-cloud, and hybrid cloud scenarios using Cilium and Istio with Helm and Operator for deployment.

In our recent post on The New Stack, we showed you how you can leverage Kubernetes (K8s) and Apache CassandraTM to manage distributed applications at scale, with thousands of nodes across both on-premises and in the cloud. In that example, we used K8ssandra and Google Cloud Platform (GCP) to illustrate some of the challenges you might expect to encounter as you grow into a multi-cloud environment, upgrade to another K8s version, or begin working with different distributions and complimentary tooling. In this post, we’ll explore a few alternative approaches to using K8s to help you more easily manage distributed applications.

5 More Reasons to Choose Apache Pulsar Over Apache Kafka

Thu, 05 May 2022 18:11:20 +0000

Author’s note: I originally published this blog post in 2019, while I was CEO of Kesque, a real-time messaging service built on Apache Pulsar, the cloud-native distributed messaging and streaming platform. It’s a follow-up to an earlier post, “ 7 Reasons to Choose Apache Pulsar over Apache Kafka.” A lot of big changes have happened since these two posts went live, including Kesque’s acquisition, in January 2021, by DataStax. The reasons to choose Pulsar, however, haven’t changed.

SQL vs. NoSQL: Pros & Cons

Thu, 21 Apr 2022 09:28:50 +0000

What’s the best way to store, protect and access your data?

This is a fundamental, yet critical decision.

After all, data is the cornerstone of success for just about every modern organization.

For most companies, the choice comes down to SQL and NoSQL databases. Each has unique strengths and weaknesses.

SQL databases have been a proven option since the 1970s. They are made up of highly structured tables, consisting of rows and columns, related to one other through common attributes. Every column is required to have a value for its corresponding row.

NoSQL (“not only SQL” or “non-SQL”) databases came along later to break the relational table straitjacket, with the ability to store and access all data types, structured and unstructured, together.

They’re extremely flexible and easy for developers to work with and modify. Learn more about SQL and NoSQL databases and their basic differences.

Full Stream Ahead: Astra Streaming, Powered by Apache Pulsar

Thu, 24 Mar 2022 09:22:15 +0000

With a serverless architecture, multi-region geo replication, and Apache Cassandra’s legendary performance, DataStax Astra DB makes it easy for developers and enterprises to start small with their applications and grow them to infinite scale without constant performance tuning and optimization exercises.

However, cloud app developers also face significant challenge`s beyond data at rest. Modern data apps require high-scale streaming technologies that can deliver the reactive engagement at the point of interaction that end users have come to expect. That’s why today, we’re excited to announce that these capabilities have been added to DataStax Astra.

Reaper 3.0 for Apache Cassandra is available

Fri, 18 Mar 2022 14:55:43 +0000

The K8ssandra team is pleased to announce the release of Reaper 3.1. Let’s dive into the features and improvements that 3.0 recently introduced (along with some notable removals) and how the newest update to 3.1 builds on that.

JDK11 support

Starting with 3.1.0, Reaper can now compile and run with jdk11. Note that jdk8 is still supported at runtime.

Storage backends

Over the years, we regularly discussed dropping support for Postgres and H2 with the The Last Pickle (TLP) team, now part of DataStax, the organization leading the open-source development of Reaper. Despite our lack of expertise in Postgres, the effort required to maintain support for these storage backends was moderate as long as Reaper’s architecture was simple. However, complexity grew with more deployment options, culminating with the addition of the sidecar mode.

Have you upgraded to Cassandra 4.0 yet?

Fri, 11 Mar 2022 08:35:20 +0000

Alongside stability, Apache Cassandra 4.0 stands out for its ability to scale operations faster, its new auditing capabilities, and the way it embraces privacy by design. In this article, we’ll cover the new features and walk you through the steps required to upgrade to Cassandra 4.0.

The general availability of Apache Cassandra 4.0 marks the most stable release in the project’s history.

Even with the 4.0 beta 1 release back in July 2020, Project Management Committee (PMC) members were confident the latest version of the distributed NoSQL database would be ready to go to work with Apache Cassandra Committer; PMC member Sankalp Kohli at the time indicated that “users will be able to use the release knowing it is production-ready on day one.”

The Serverless Database You Really Want

Thu, 03 Mar 2022 20:43:05 +0000

The dreaded part of every site reliability engineer’s (SRE) job eventually: capacity planning. You know, the dance between all the stakeholders when deploying your applications. Did engineering really simulate the right load and do we understand how the application scales? Did product managers accurately estimate the amount of usage? Did we make architectural decisions that will keep us from meeting our SLA goals? And then the question that everyone will have to answer eventually: how much is this going to cost? This forces SREs to assume the roles of engineer, accountant and fortune teller.

Save Hours on your Setup of K8ssandra with the New Config Builder

Thu, 24 Feb 2022 19:36:47 +0000

Setting up K8ssandra in your workflow just got a whole lot easier. With the new Config Builder you can be running Apache Cassandra® on Kubernetes in a matter of minutes.

The purpose of K8ssandra is to make it easy to run Apache Cassandra® on Kubernetes. We recently took another big step in that direction by releasing the K8ssandra config builder.

Even if you’ve created thousands of nodes or integrated K8ssandra in your stack, you’ll probably want to give the config builder a try. Walk through the interactive wizard defining the shape of your cluster, resource requirements, and toggle features to fit your needs. This is great for smoothing out the on-ramp for a production ready environment of K8ssandra.

Why we decided to build a K8ssandra Operator - Part 4

Tue, 22 Feb 2022 17:04:28 +0000

In the first, second, and third posts in this series, we’ve shared conversations with K8ssandra core team members on our journey to build a Kubernetes operator for K8ssandra. We’ve discussed the virtues of the Helm package manager versus Kubernetes operators for deploying and managing infrastructure in Kubernetes and some of our implementation choices for the operator.

In this final post of the series, we pick up from the previous post with a discussion of how we decided to structure our projects in GitHub, how we test the K8ssandra operator, and our hopes for how the operator will expand the K8ssandra developer community.

Cassandra Database Migration to Kubernetes with Zero Downtime

Tue, 15 Feb 2022 15:22:15 +0000

K8ssandra is a cloud-native distribution of the Apache Cassandra® database that runs on Kubernetes, with a suite of tools to ease and automate operational tasks. In this post, we’ll walk you through a database migration from a Cassandra cluster running in AWS EC2 to a K8ssandra cluster running in Kubernetes on AWS EKS, with zero downtime.

As an Apache Cassandra user, your expectation should be that migrating to K8ssandra would happen without downtime. To make that happen with “classic” clusters running on virtual machines or bare metal instances, you will use the datacenter (DC) switch technique which is commonly used in the Cassandra community to transfer clusters to different hardware or environments. The good news is that it’s not very different for clusters running in Kubernetes as most Container Network Interfaces (CNI) will provide routable pod IPs.

Building Reactive Java Applications with Spring Framework

Tue, 15 Feb 2022 14:09:28 +0000

In one of our many free tutorials on DataStax Developers YouTube channel, we walked you through how to build a reactive implementation of Spring PetClinic in Apache Cassandra® using Spring WebFlux. The full series is available on YouTube.

If you’re a Java developer who uses the Spring ecosystem, you’ve probably seen the Spring Pet Clinic. In this workshop, we will walk you through a new reactive implementation of the Pet Clinic backend that uses Spring WebFlux and Apache Cassandra® (via DataStax Astra DB).

SQL vs. NoSQL: Pros & Cons

Wed, 09 Feb 2022 09:26:22 +0000

What’s the best way to store, protect and access your data? This is a fundamental, yet critical decision. After all, data is the cornerstone of success for just about every modern organization. For most companies, the choice comes down to SQL and NoSQL databases. Each has unique strengths and weaknesses.

SQL databases have been a proven option since the 1970s. They are made up of highly structured tables, consisting of rows and columns, related to one other through common attributes. Every column is required to have a value for its corresponding row. NoSQL (“not only SQL” or “non-SQL”) databases came along later to break the relational table straitjacket, with the ability to store and access all data types, structured and unstructured, together. They’re extremely flexible and easy for developers to work with and modify. Learn more about SQL and NoSQL databases and their basic differences.