question

salmanwelji avatar image
salmanwelji asked phantomreference82 commented

why cant i fetch more then 1000 record in one call

i know this can be solved using off set but doing that it makes the job very slow. i can do off set for incremental calls once i fetch the complete data.
is there a way to fetch one time everything fast?
i'm using python to fetch and then dumping to SQL but i need to make recursive calls to fetch related line item detail. im really stuck with the time its taking for only data from September 2018 it took 12hours. if anyone can help.
REST API
4 comments
10 |2000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Jeffrey Blattman avatar image Jeffrey Blattman ♦♦ commented ·

What endpoints are you using to fetch data?

0 Likes 0 ·
salmanwelji avatar image salmanwelji Jeffrey Blattman ♦♦ commented ·

hi jeffrey, im calling to get order list first and based on that order list im trying to fetch lineitem in both cases i have to do off set. fetching orderlist its not that delay but when making recursive calls to get line item based on order list its taking long time like hours

0 Likes 0 ·
Jeffrey Blattman avatar image Jeffrey Blattman ♦♦ salmanwelji commented ·

Why do you need a list of orders going back months?

0 Likes 0 ·
phantomreference82 avatar image phantomreference82 commented ·

Assuming that you're hitting the endpoint '{url}/v3/merchants/{merchantId}/items',
items past the 1000-item limit can be retrieved by appending '?limit=999&offset=n' to the URL.

For the next 999 items, set offset to 1... For the 999 items after that, set offset to 2... And so on

0 Likes 0 ·

1 Answer

Jeffrey Blattman avatar image
Jeffrey Blattman answered salmanwelji commented
You are deliberately hobbled here. Querying large amounts of data like this, especially historical data that wouldn't otherwise be cached, puts an extreme load on our server. If you use our normal API endpoints for this regularly it's likely you'll be cut off at some point.

You should probably start with a description of why you want to do what you are doing.

Here is a beta API designed for data export:
https://github.com/clover/export-api-examples


1 comment
10 |2000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

salmanwelji avatar image salmanwelji commented ·

i understand, but i need at least 3month of historic data to create some analysis based on it then once i have that ill be doing incremental load everyday or every week but for now i need historic data.

the link you sent will be helpful in making recursive calls?

0 Likes 0 ·

Welcome to the
Clover Developer Community