Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions antora.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,3 +12,8 @@ asciidoc:
pulsar-version: '2.10'
admin-console-version: '2.0.4'
heartbeat-version: '1.0.12'
starlight-kafka: 'Starlight for Kafka'
starlight-rabbitmq: 'Starlight for RabbitMQ'
pulsar-reg: 'Apache Pulsar(TM)'
pulsar: 'Apache Pulsar'
pulsar-short: 'Pulsar'
3 changes: 1 addition & 2 deletions modules/ROOT/nav.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,7 @@
* xref:components:heartbeat-vm.adoc[]
* xref:components:pulsar-beam.adoc[]
* xref:components:pulsar-sql.adoc[]
* xref:components:starlight-for-kafka.adoc[]
* xref:components:starlight-for-rabbitmq.adoc[]
* xref:components:starlight.adoc[]

.Operations
* xref:operations:auth.adoc[]
Expand Down
107 changes: 47 additions & 60 deletions modules/ROOT/pages/faqs.adoc
Original file line number Diff line number Diff line change
@@ -1,123 +1,110 @@
= Luna Streaming FAQs
:navtitle: FAQs

If you are new to DataStax Luna Streaming and its Apache Pulsar enhancements, these FAQs are for you.
If you are new to {company} Luna Streaming and its {pulsar} enhancements, these FAQs are for you.

== Introduction
== What is {company} Luna Streaming?

=== What is DataStax Luna Streaming?
{company} Luna Streaming is a new Kubernetes-based distribution of {pulsar}, based on the technology that https://kesque.com/[Kesque] built to run its {pulsar-short}-as-a-service.

DataStax Luna Streaming is a new Kubernetes-based distribution of Apache Pulsar, based on the technology that https://kesque.com/[Kesque] built to run its Pulsar-as-a-service.
== What components and features are provided by {company} Luna Streaming?

=== What components and features are provided by DataStax Luna Streaming?

In addition to Apache Pulsar itself, DataStax Luna Streaming provides:
In addition to {pulsar} itself, {company} Luna Streaming provides:

* An installer that can stand up a dev or production cluster on bare metal or VMs without a pre-existing Kubernetes environment
* A helm chart that can deploy and manage Pulsar on your current Kubernetes infrastructure
* A Helm chart that can deploy and manage {pulsar-short} on your current Kubernetes infrastructure
* Cassandra, Elastic, Kinesis, Kafka, and JDBC connectors
* A management dashboard
* A monitoring and alerting system

=== On which version of Apache Pulsar is DataStax Luna Streaming based?
== On which version of {pulsar} is {company} Luna Streaming based?

DataStax Luna Streaming {luna-version} is based on its distribution of Apache Pulsar {pulsar-version}, plus features and additional enhancements from DataStax contributors.
{company} Luna Streaming {luna-version} is based on its distribution of {pulsar} {pulsar-version}, plus features and additional enhancements from {company} contributors.

=== What does DataStax Luna Streaming provide that I cannot get with open-source Apache Pulsar?
== What does {company} Luna Streaming provide that I cannot get with open-source {pulsar}?

DataStax Luna Streaming is a hardened version of Apache Pulsar that been run through additional testing to ensure it is ready for production use. It also includes additional tooling to help monitor your system, including an enhanced Admin Console and a Heartbeat service to monitor the system health.
{company} Luna Streaming is a hardened version of {pulsar} that been run through additional testing to ensure it is ready for production use. It also includes additional tooling to help monitor your system, including an enhanced Admin Console and a Heartbeat service to monitor the system health.

=== Is DataStax Luna Streaming an open-source project?
== Is {company} Luna Streaming an open-source project?

Yes, DataStax Luna Streaming is open source. See the <<gitHubRepos,repos FAQ>>.
Yes, {company} Luna Streaming is open source. See the <<gitHubRepos,repos FAQ>>.

=== Which Kubernetes platforms are supported by DataStax Luna Streaming?
== Which Kubernetes platforms are supported by {company} Luna Streaming?

They include Minikube, K8d, Kind, Google Kubernetes Engine (GKE), Microsoft Azure Kubernetes Service (AKS), Amazon Elastic Kubernetes Service (EKS), and other commonly used platforms.

[#gitHubRepos]
=== Where are the DataStax Luna Streaming public GitHub repos?
== Where are the {company} Luna Streaming public GitHub repos?

There are several public repos, each with a different purpose. See:

* https://github.com/datastax/pulsar[https://github.com/datastax/pulsar] : This is the distro repo (a fork of apache/pulsar).
* https://github.com/datastax/pulsar-admin-console[https://github.com/datastax/pulsar-admin-console] : This is the repo for the Pulsar admin console, which allows for the configuration and monitoring of Pulsar.
* https://github.com/datastax/pulsar-heartbeat[https://github.com/datastax/pulsar-heartbeat] : This is a monitoring/observability tool for Pulsar that tracks the health of the cluster and can generate alerts in Slack and OpsGenie.
* https://github.com/datastax/pulsar-helm-chart[https://github.com/datastax/pulsar-helm-chart] : This is the Helm chart for deploying the DataStax Pulsar Distro in an existing Kubernetes cluster.
* https://github.com/datastax/pulsar-sink[https://github.com/datastax/pulsar-sink] : This is the DataStax Apache Pulsar Connector (`pulsar-sink` for Cassandra) repo.
* https://github.com/datastax/burnell[https://github.com/datastax/burnell] : This is a utility for Pulsar that provides various functions, such as key initialization for authentication, and JWT token creation API.

== Installation
* https://github.com/datastax/pulsar-admin-console[https://github.com/datastax/pulsar-admin-console] : This is the repo for the {pulsar-short} admin console, which allows for the configuration and monitoring of {pulsar-short}.
* https://github.com/datastax/pulsar-heartbeat[https://github.com/datastax/pulsar-heartbeat] : This is a monitoring/observability tool for {pulsar-short} that tracks the health of the cluster and can generate alerts in Slack and OpsGenie.
* https://github.com/datastax/pulsar-helm-chart[https://github.com/datastax/pulsar-helm-chart] : This is the Helm chart for deploying the {company} {pulsar-short} Distro in an existing Kubernetes cluster.
* https://github.com/datastax/pulsar-sink[https://github.com/datastax/pulsar-sink] : This is the {company} {pulsar} Connector (`pulsar-sink` for Cassandra) repo.
* https://github.com/datastax/burnell[https://github.com/datastax/burnell] : This is a utility for {pulsar-short} that provides various functions, such as key initialization for authentication, and JWT token creation API.

=== Is there a prerequisite version of Java needed for the DataStax Luna Streaming installation?
== Is there a prerequisite version of Java needed for the {company} Luna Streaming installation?

The DataStax Luna Streaming distribution is designed for Java 11. However, because the product releases Docker images, you do not need to install Java (8 or 11) in advance. Java 11 is bundled in the Docker image.
The {company} Luna Streaming distribution is designed for Java 11. However, because the product releases Docker images, you do not need to install Java (8 or 11) in advance. Java 11 is bundled in the Docker image.

=== What are the install options for DataStax Luna Streaming?
== What are the install options for {company} Luna Streaming?

* Use the Helm chart provided at https://github.com/apache/pulsar-helm-chart[https://github.com/datastax/pulsar-helm-chart] to install DataStax Luna Streaming in an existing Kubernetes cluster on your laptop or hosted by a cloud provider.
* Use the tarball provided at https://github.com/datastax/pulsar/releases[https://github.com/datastax/pulsar/releases] to install DataStax Luna Streaming on a server or VM.
* Use the DataStax Ansible scripts provided at https://github.com/datastax/pulsar-ansible[https://github.com/datastax/pulsar-ansible] to install DataStax Luna Streaming on a server or VM with our provided playbooks.
* Use the Helm chart provided at https://github.com/apache/pulsar-helm-chart[https://github.com/datastax/pulsar-helm-chart] to install {company} Luna Streaming in an existing Kubernetes cluster on your laptop or hosted by a cloud provider.
* Use the tarball provided at https://github.com/datastax/pulsar/releases[https://github.com/datastax/pulsar/releases] to install {company} Luna Streaming on a server or VM.
* Use the {company} Ansible scripts provided at https://github.com/datastax/pulsar-ansible[https://github.com/datastax/pulsar-ansible] to install {company} Luna Streaming on a server or VM with our provided playbooks.

=== How do I install DataStax Luna Streaming in my Kubernetes cluster?
== How do I install {company} Luna Streaming in my Kubernetes cluster?

Follow the full instructions in xref:install-upgrade:quickstart-helm-installs.adoc[Quick Start for Helm Chart installs].

=== How do I install DataStax Luna Streaming on my server or VM?
== How do I install {company} Luna Streaming on my server or VM?

Follow the full instructions in xref:install-upgrade:quickstart-server-installs.adoc[Quick Start for Server/VM installs].

== What task can I perform in the DataStax Luna Streaming Admin Console?
== What task can I perform in the {company} Luna Streaming Admin Console?

From the Admin Console, you can:

* Add and run Pulsar clients
* Add and run {pulsar-short} clients
* Establish credentials for secure connections
* Define topics that can be published for streaming apps
* Set up Pulsar sinks that publish topics and make them available to subscribers, such as for a Cassandra database table
* Control namespaces used by Pulsar
* Set up {pulsar-short} sinks that publish topics and make them available to subscribers, such as for a Cassandra database table
* Control namespaces used by {pulsar-short}
* Use the Admin API

== What is Pulsar Heartbeat?

https://github.com/datastax/pulsar-heartbeat[Pulsar Heartbeat] monitors the availability, tracks the performance, and reports failures of the Pulsar cluster. It produces synthetic workloads to measure end-to-end message pubsub latency. Pulsar Heartbeat is a cloud-native application that can be installed by Helm within the Pulsar Kubernetes cluster.

== What is Prometheus?
== What is {pulsar-short} Heartbeat?

https://prometheus.io/docs/introduction/overview/[Prometheus] is an open-source tool to collect metrics on a running app, providing real-time monitoring and alerts.
https://github.com/datastax/pulsar-heartbeat[{pulsar-short} Heartbeat] monitors the availability, tracks the performance, and reports failures of the {pulsar-short} cluster. It produces synthetic workloads to measure end-to-end message pubsub latency. {pulsar-short} Heartbeat is a cloud-native application that can be installed by Helm within the {pulsar-short} Kubernetes cluster.

== What is Grafana?
== What are the features provided by {company} {pulsar} Connector (`pulsar-sink`) that are not supported in `kafka-sink`?

https://grafana.com/[Grafana] is a visualization tool that helps you make sense of metrics and related data coming from your apps via Prometheus, for example.
The https://pulsar.apache.org/docs/en/io-overview/[{pulsar-short} IO framework] provides many features that are not possible in Kafka, and has different compression formats and auth/security features. The features are handled by {pulsar-short}. For more, see xref:operations:io-connectors.adoc[Luna Streaming IO Connectors].

== Pulsar Connector
The {company} {pulsar} Connector allows single-record acknowledgement and negative acknowledgements.

=== What are the features provided by DataStax Apache Pulsar Connector (`pulsar-sink`) that are not supported in `kafka-sink`?

The https://pulsar.apache.org/docs/en/io-overview/[Pulsar IO framework] provides many features that are not possible in Kafka, and has different compression formats and auth/security features. The features are handled by Pulsar. For more, see xref:operations:io-connectors.adoc[Luna Streaming IO Connectors].

The DataStax Apache Pulsar Connector allows single-record acknowledgement and negative acknowledgements.

=== What features are missing in DataStax Apache Pulsar Connector (`pulsar-sink`) compared with `kafka-sink`?
== What features are missing in {company} {pulsar} Connector (`pulsar-sink`) compared with `kafka-sink`?

* No support for `tinyint` (`int8bit`) and `smallint` (`int16bit`).
* The key is always a String, but you can write JSON inside it; the support is implemented in pulsar-sink, but not in Pulsar IO.
* The key is always a String, but you can write JSON inside it; the support is implemented in pulsar-sink, but not in {pulsar-short} IO.
* The “value” of a “message property” is always a String; for example, you cannot map the message property to `__ttl` or to `__timestamp`.
* Field names inside structures must be valid for Avro, even in case of JSON structures. For example, field names like `Int.field` (with dot) or `int field` (with space) are not valid.

=== How is DataStax Apache Pulsar Connector distributed?
== How is {company} {pulsar} Connector distributed?

There are two packages:

* The `pulsar-sink` functionality of DataStax Apache Pulsar Connector is included with DataStax Luna Streaming. It's built in!
* You can optionally download the DataStax Apache Pulsar Connector tarball from the https://downloads.datastax.com/#pulsar-sink[DataStax Downloads] site, and then use it as its own product with your open-source Apache Pulsar install.

If you're using open-source software (OSS) Apache Pulsar, you can use DataStax Apache Pulsar Connector with the OSS to take advantage of this `pulsar-sink` for Cassandra. See the xref:pulsar-connector:ROOT:index.adoc[DataStax Apache Pulsar Connector documentation].
* The `pulsar-sink` functionality of {company} {pulsar} Connector is included with {company} Luna Streaming. It's built in!
* You can optionally download the {company} {pulsar} Connector tarball from the https://downloads.datastax.com/#pulsar-sink[{company} Downloads] site, and then use it as its own product with your open-source {pulsar} install.

== APIs
If you're using open-source software (OSS) {pulsar}, you can use {company} {pulsar} Connector with the OSS to take advantage of this `pulsar-sink` for Cassandra. See the xref:pulsar-connector:ROOT:index.adoc[{company} {pulsar} Connector documentation].

=== What client APIs does DataStax Luna Streaming provide?
== What is the {company} Change Data Capture (CDC) for Cassandra connector?

The same as for Apache Pulsar. See https://pulsar.apache.org/docs/en/client-libraries/.
This source connector streams data changes from Cassandra tables to Pulsar topics.
For more information, see the xref:cdc-for-cassandra:ROOT:index.adoc[{company} CDC for Cassandra connector documentation].

== What client APIs does {company} Luna Streaming provide?

The same as for {pulsar}. See https://pulsar.apache.org/docs/en/client-libraries/.
20 changes: 10 additions & 10 deletions modules/ROOT/pages/index.adoc
Original file line number Diff line number Diff line change
@@ -1,27 +1,27 @@
= Welcome to DataStax Luna Streaming
= Welcome to {company} Luna Streaming
:navtitle: Luna Streaming

DataStax Luna Streaming is a production-ready distribution of Apache Pulsar built to run seamlessly on any CNCF conformant version of Kubernetes. DataStax Luna Streaming provides all of the core capabilities included in the Apache Community version of Apache Pulsar, plus a number of additional tools and features to facilitate administration and operational tasks associated with running Apache Pulsar in production.
{company} Luna Streaming is a production-ready distribution of {pulsar} built to run seamlessly on any CNCF conformant version of Kubernetes. {company} Luna Streaming provides all of the core capabilities included in the Apache Community version of {pulsar}, plus a number of additional tools and features to facilitate administration and operational tasks associated with running {pulsar} in production.

== Release notes

The latest release of DataStax Luna Streaming is {luna-version}, which matches the supported, distributed Pulsar version numbers.
The latest release of {company} Luna Streaming is {luna-version}, which matches the supported, distributed {pulsar-short} version numbers.

The prior Luna Streaming release (numbered 1.0.x or 2.7.2) provided the Pulsar 2.7.2 distribution.
The prior Luna Streaming release (numbered 1.0.x or 2.7.2) provided the {pulsar-short} 2.7.2 distribution.

Refer to the DataStax Luna Streaming https://github.com/datastax/release-notes/blob/master/Luna_Streaming_2.8_Release_Notes.md[release notes], which are hosted in our public GitHub repo, for information &amp; linked commit IDs that were implemented in the latest Luna Streaming {luna-version} release.
Refer to the {company} Luna Streaming https://github.com/datastax/release-notes/blob/master/Luna_Streaming_2.8_Release_Notes.md[release notes], which are hosted in our public GitHub repo, for information &amp; linked commit IDs that were implemented in the latest Luna Streaming {luna-version} release.

== Components

In addition to the distribution of https://pulsar.apache.org/en/versions/[Apache Pulsar {pulsar-version}], DataStax Luna Streaming provides:
In addition to the distribution of https://pulsar.apache.org/en/versions/[{pulsar} {pulsar-version}], {company} Luna Streaming provides:

* A xref:install-upgrade:quickstart-helm-installs.adoc[Helm chart] that deploys and manages Pulsar on your current CNCF-conformant Kubernetes infrastructure
* A xref:install-upgrade:quickstart-helm-installs.adoc[Helm chart] that deploys and manages {pulsar-short} on your current CNCF-conformant Kubernetes infrastructure

* Cassandra, Elastic, Kinesis, Kafka, and JDBC xref:operations:io-connectors.adoc[connectors]

* xref:components:admin-console-vm.adoc[Pulsar Admin Console] for simplified administration of your Pulsar environment
* xref:components:admin-console-vm.adoc[{pulsar-short} Admin Console] for simplified administration of your {pulsar-short} environment

* xref:components:heartbeat-vm.adoc[Pulsar Heartbeat] to observe and monitor your Pulsar instances
* xref:components:heartbeat-vm.adoc[{pulsar-short} Heartbeat] to observe and monitor your {pulsar-short} instances

== Features

Expand Down Expand Up @@ -57,5 +57,5 @@ In addition to the distribution of https://pulsar.apache.org/en/versions/[Apache

* If you have an existing Kubernetes environment, deploy Luna Streaming with a xref:install-upgrade:quickstart-helm-installs.adoc[Helm Installation].
* If you have a bare metal or a cloud environment, see xref:install-upgrade:quickstart-server-installs.adoc[Server/VM Installation].
* If you want to learn about monitoring with Pulsar Heartbeat, see xref:components:pulsar-monitor.adoc[Pulsar Heartbeat].
* If you want to learn about monitoring with {pulsar-short} Heartbeat, see xref:components:pulsar-monitor.adoc[{pulsar-short} Heartbeat].
* If you have questions about Luna Streaming, see xref::faqs.adoc[Luna Streaming FAQs].
4 changes: 2 additions & 2 deletions modules/ROOT/partials/install-helm.adoc
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
. Add the DataStax Helm chart repo to your Helm store:
. Add the {company} Helm chart repo to your Helm store:
+
[source,shell]
----
helm repo add datastax-pulsar https://datastax.github.io/pulsar-helm-chart
----

. Install the Helm chart using a minimalist values file.
This command creates a Helm release named "my-pulsar-cluster" using the DataStax Luna Helm chart, within the K8s namespace "datastax-pulsar".
This command creates a Helm release named "my-pulsar-cluster" using the {company} Luna Helm chart, within the K8s namespace "datastax-pulsar".
The minimal cluster creates only the essential components and has no ingress or load balanced services.
+
[source,shell,subs="attributes+"]
Expand Down
2 changes: 1 addition & 1 deletion modules/ROOT/partials/manually-create-credentials.adoc
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@

A number of values need to be stored in secrets prior to enabling token-based authentication.

. Generate a key-pair for signing the tokens using the Pulsar tokens command:
. Generate a key-pair for signing the tokens using the {pulsar-short} tokens command:
+
[source,bash]
----
Expand Down
6 changes: 0 additions & 6 deletions modules/ROOT/partials/port-forward-web-service.adoc

This file was deleted.

Loading