when filtering on an attribute, it doesn't return all the qualifying records. it's really annoying. it's not because of any filter limitation as i have set that to a very high number. i only have a couple of hundred records in my dynamodb table. any idea why is it happening? found a lot of posts about this on forums. Thanks
Hi Eugene, There could be a couple things happening here that require you to understand how dynamodb works behind the scenes. When you perform a query or scan, dynamodb returns one page of results with 1mb worth of data. It will then apply your filter to the 1mb page. If there is more data, you will need to paginate and make subsequent queries to get all your results. Check out these videos for more about pagination: th-cam.com/video/i5twM5y8ATU/w-d-xo.html th-cam.com/video/Ifcic-JIw1k/w-d-xo.html Cheers, Daniel
Hello, thank you for the video! I am encountering an error while using response = table.scan(FilterExpression= Attr('DataTransfer').eq(DataTransfer)) { "errorMessage": "name 'Attr' is not defined", "errorType": "NameError", "stackTrace": [ " File \"/var/task/lambda_function.py\", line 175, in lambda_handler response = table.scan(FilterExpression= Attr('DataTransfer').eq(DataTransfer)) " ] } As you said it should be pretty straight forward to query this, but I am not able to figure out why is Attr not defined. Thanks in advance!
Does it make sense to populating a new table to avoid scan operation here? e.g. to have dynamo DB stream of this table to populate another table with partition key OriginCountry and value being the set of AccountId? Just wondering if it's a reasonable approach for NoSQL, great video btw
Excellent series!
Hello, Thanks for video.
How to use document mode in dynamoDB.
Really clean video
Thanks!
when filtering on an attribute, it doesn't return all the qualifying records. it's really annoying. it's not because of any filter limitation as i have set that to a very high number. i only have a couple of hundred records in my dynamodb table. any idea why is it happening? found a lot of posts about this on forums. Thanks
Hi Eugene,
There could be a couple things happening here that require you to understand how dynamodb works behind the scenes. When you perform a query or scan, dynamodb returns one page of results with 1mb worth of data. It will then apply your filter to the 1mb page. If there is more data, you will need to paginate and make subsequent queries to get all your results.
Check out these videos for more about pagination:
th-cam.com/video/i5twM5y8ATU/w-d-xo.html
th-cam.com/video/Ifcic-JIw1k/w-d-xo.html
Cheers,
Daniel
@@BeABetterDev thanks a lot!!
Hello, thank you for the video!
I am encountering an error while using
response = table.scan(FilterExpression= Attr('DataTransfer').eq(DataTransfer))
{
"errorMessage": "name 'Attr' is not defined",
"errorType": "NameError",
"stackTrace": [
" File \"/var/task/lambda_function.py\", line 175, in lambda_handler
response = table.scan(FilterExpression= Attr('DataTransfer').eq(DataTransfer))
"
]
}
As you said it should be pretty straight forward to query this, but I am not able to figure out why is Attr not defined. Thanks in advance!
try adding below import statement. It should resolve your issue -
from boto3.dynamodb.conditions import Attr
Thanks; need demo of conditionalexpression while PutItem method.
Does it make sense to populating a new table to avoid scan operation here? e.g. to have dynamo DB stream of this table to populate another table with partition key OriginCountry and value being the set of AccountId? Just wondering if it's a reasonable approach for NoSQL, great video btw
You can use DynamDB GSI to do that. GSI just replicate the same data but use OriginCountry as Partition Key.
i just see scan where is .query
*Constant time