Do not see data in AWS Kinesis

47 Views Asked by At

I created a stream in AWS Kinesis, straightforward: enter image description here

I put records into the stream (I took the script from AWS documentation):

import datetime
import json
import random
import boto3

STREAM_NAME = "apache-flink-demo-input-1"


def get_data():
    return {
        'event_time': datetime.datetime.now().isoformat(),
        'ticker': random.choice(['AAPL', 'AMZN', 'MSFT', 'INTC', 'TBV']),
        'price': round(random.random() * 100, 2)}


def generate(stream_name, kinesis_client):
    while True:
        data = get_data()
        print(data)
        response = kinesis_client.put_record(
            StreamName=stream_name,
            Data=json.dumps(data),
            PartitionKey="1")
        print(response)
        print()


if __name__ == '__main__':
    generate(STREAM_NAME,
             boto3.client(
                 'kinesis',
                 aws_access_key_id="XXXX", # Hidden
                 aws_secret_access_key="XXXX",  # Hidden
                 region_name='us-west-1'))

I got a response that it's successful for every record:

{'ShardId': 'shardId-000000000003', 'SequenceNumber': '49644599907995286627749093391250621075271860022832791602', 'ResponseMetadata': {'RequestId': 'f0e04f67-94fa-934b-ad60-6e3e40d06706', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': 'f0e04f67-94fa-934b-ad60-6e3e40d06706', 'x-amz-id-2': 'KLepCUNwgrZMY639W6QhdyFRsfeiqAnBqdLWdPSdDKxgDnGUqRGNly/dCPE/GHGVhUN3YXXB98PUvtWRHidOM4PRI4/edeaI', 'date': 'Sat, 16 Sep 2023 21:33:13 GMT', 'content-type': 'application/x-amz-json-1.1', 'content-length': '110'}, 'RetryAttempts': 0}}

But no data in the dashboard: enter image description here

1

There are 1 best solutions below

1
Rakshith On

Looks like you are looking at the wrong stream

STREAM_NAME = "apache-flink-demo-input-1"

However the stream which you have posted in the screenshot is the output stream. You may want to check the appropriate stream.