Helper function for pagination using AWS boto3#

I noticed that a lot of my boto3 code in s3-credentials looked like this:

1
paginator = iam.get_paginator("list_user_policies")
2
for response in paginator.paginate(UserName=username):
3
    for policy_name in response["PolicyNames"]:
4
        print(policy_name)

This was enough verbosity that I was hesitating on implementing pagination properly for some method calls.

I came up with this helper function to use instead:

1
def paginate(service, method, list_key, **kwargs):
2
    paginator = service.get_paginator(method)
3
    for response in paginator.paginate(**kwargs):
4
        yield from response[list_key]

Now the above becomes:

1
for policy_name in paginate(iam, "list_user_policies", "PolicyNames", UserName=username):
2
    print(policy_name)

Here’s the issue and the refactoring commit.

Helper function for pagination using AWS boto3

https://mranv.pages.dev/posts/helper-function-for-pagination-using-aws-boto3/

Author

Anubhav Gain

Published at

2024-06-07

License

CC BY-NC-SA 4.0

Interpreting photo orientation and locations in EXIF data

Testing things in Fedora using Docker

Helper function for pagination using AWS boto3