Topic Partitions

Partitions, which exist within topics, are parallel, ordered, immutable sequences of messages that are continually appended to.

Topics can contain multiple partitions, which make topics scalable by spreading the load for a topic across multiple servers.

Downstream applications that read messages can read from multiple partitions within a topic for faster performance than would be possible if they read from a single partition per topic. Downstream applications can also scale by having separate instances read from separate partitions.

When creating or editing a stream, a default number of partitions can be specified for that stream's topics. Topics inherit the stream's partition default. However, topics can also override the stream's partition default by setting the number of partitions to be used.

Performance

The default number of partitions for data-fabric streams and topics can impact performance. Depending on the volume of messages being published to a topic, the default number of partitions might be increased for efficient consumption.

When there is a high volume of messages being published to a topic:

Multiple consumers, in consumer groups, reading from multiple partitions are handled more efficiently.
Individual consumers each reading from a single partition are handled less efficiently.

Reference

The following lists topics that have more detailed information.

See the maprcli stream create for information about creating streams with the -defaultpartitions parameter.
See the maprcli stream edit for information about editing streams with the -defaultpartitions parameter.
See the maprcli stream topic create for information about creating topics with the -partitions parameter.
See the maprcli stream topic edit for information about modifying topics with the -partitions parameter.
See the maprcli stream topic info for information about topic data including the -partitions parameter.
See the HPE Ezmeral Data Fabric Event Store Java API Library for the methods used to create and edit streams and to create and edit topics.