Metadata API

List Topics

GET /topics

Lists all topics belonging to the user.

Response:

JSON Map of topic name to topic’s partition:

{
  "cities": 12,
  "greetings": 1,
  "words": 137
}

List Consumers

• GET /consumers

Lists consumers belonging to the user known by the REST server.

Response:

JSON array of GroupAssignments.

TopicAssignments{
    topic: String,
    partitions: Array<Int>
}

InstanceAssignments{
    name: String,
    topics: Array<TopicAssignments>
}

GroupAssignments{
    name: String,
    instances: Array<InstanceAssignments>
}

[
  {
    "name": "mygroup",
    "instances": [
      {
        "name": "instance-1",
        "topics": [
          {
            "topic": "cities",
            "partitions": [0, 1, 2]
          },
          {
            "topic": "words",
            "partitions": [10, 21, 32]
          }
        ]
      },
      {
        "name": "instance-2",
        "topics": [
          {
            "topic": "cities",
            "partitions": [3, 4, 5]
          },
          {
            "topic": "words",
            "partitions": [1, 3, 5, 7]
          }
        ]
      }
    ]
  }
]

List Committed Offsets

[GET | POST] /committed/$CONSUMER_GROUP/$INSTANCE_ID?timeout=$TIMEOUT

Returns the last committed offsets for the topic partitions inside the group. Can be used alongside Commit Consumer API. Request body should be a single TopicPartition object or an array of TopicPartitions:

TopicPartition{topic: String, partition: Int}

timeout parameter defines the time to wait at most for the offsets in milliseconds. It’s optional and its default value is 10 seconds (10000).

• List committed offset for a single topic partition:

curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/committed/mygroup/myconsumer -u myuser:mypass \
    -d '{"topic": "cities", "partition": 0}'

• List committed offsets for multiple topic partitions:

curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/committed/mygroup/myconsumer -u myuser:mypass \
    -d '[
        {"topic": "cities", "partition": 0},
        {"topic": "cities", "partition": 1},
        {"topic": "greetings", "partition": 0}
    ]'

• List committed offsets with a 1 second timeout:

curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/committed/mygroup/myconsumer?timeout=1000 -u myuser:mypass \
    -d '{"topic": "cities", "partition": 1}'

Response:

Returns a JSON array of TopicPartitionOffsets:

[
  { "topic": "cities", "partition": 0, "offset": 11 },
  { "topic": "cities", "partition": 1, "offset": 21 },
  { "topic": "greetings", "partition": 0, "offset": 1 }
]

Get Topic Partition Offsets

[GET | POST] /offsets/$TIMESTAMP?timeout=$TIMEOUT

Returns the offsets for the given partitions by timestamp. The returned offset for each partition is the earliest offset whose timestamp is greater than or equal to the given timestamp in the corresponding partition.

Request body should be a single TopicPartition object or an array of TopicPartitions:

TopicPartition{topic: String, partition: Int}

timeout parameter defines the time to wait at most for the offsets in milliseconds. It’s optional and its default value is 10 seconds (10000).

• Offset for a single topic partition:

  curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/offsets/1642591892 -u myuser:mypass \
      -d '{"topic": "cities", "partition": 0}'
  

• Offsets for multiple topic partitions:

  curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/offsets/1642591892 -u myuser:mypass \
      -d '[
          {"topic": "cities", "partition": 0},
          {"topic": "cities", "partition": 1},
          {"topic": "greetings", "partition": 0}
      ]'
  

To find out the first and the end offsets for the partitions, earliest and latest strings should be used as timestamp values.

• Beginning offsets for topic partitions:

  curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/offsets/earliest -u myuser:mypass \
      -d '[
          {"topic": "cities", "partition": 0},
          {"topic": "cities", "partition": 1},
          {"topic": "greetings", "partition": 0}
      ]'
  

• End offsets for topic partitions:

  curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/offsets/latest -u myuser:mypass \
      -d '[
          {"topic": "cities", "partition": 0},
          {"topic": "cities", "partition": 1},
          {"topic": "greetings", "partition": 0}
      ]'
  

• Offsets with a 1 second timeout:

  curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/offsets/latest?timeout=1000 -u myuser:mypass \
      -d '{"topic": "cities", "partition": 1}'
  

Response:

Returns a JSON array of TopicPartitionOffsets:

[
  { "topic": "cities", "partition": 0, "offset": 11 },
  { "topic": "cities", "partition": 1, "offset": 21 },
  { "topic": "greetings", "partition": 0, "offset": 1 }
]