500 error on BAG V1 API verblijfsobjecten endpoint

Hi,

I also encountered a similar problem of using POST to url: https://bag.basisregistraties.overheid.nl/api/v1/verblijfsobjecten. with payload geometry.

I got 500 error code every single time, when I am fetching a lot of verblijfsobjecten, few thousands of objects.

Here is the error code:
{"timestamp":1606218059087,"path":"/verblijfsobjecten","status":500,"error":"Internal Server Error","message":"An error occured from which the server was unable to recover. Please contact the API maintainer with the following details: '5e646592-e2bf-4565-901b-3860d82eaabb'","requestId":"61fdf882-9667390"}

Can someone please help me with this? any suggestions?

A time limit is exceeded in the backend, but I have a hard time reproducing it.

What is the exact request you do when you get the 500 error? Query parameters and request body?

Hi Robin,

Here is my request details:

curl --location --request POST 'https://bag.basisregistraties.overheid.nl/api/v1/verblijfsobjecten?pageSize=100' \
--header 'x-api-key: MY_API_KEY' \
--header 'Content-Type: application/json' \
--data-raw '{"geometrie":{"within":{"type":"Polygon","coordinates":[[[4.342760743,52.09161594],[4.350757654,52.084730814],[4.34881838,52.083869624],[4.348118488,52.082960075],[4.344548425,52.081458674],[4.344460114,52.081170192],[4.343667582,52.080823105],[4.344745401,52.079867259],[4.341653149,52.078547576],[4.338358697,52.077184899],[4.336916929,52.076520856],[4.335706568,52.076203782],[4.334384947,52.07603884],[4.332786564,52.075981528],[4.330933403,52.076802083],[4.326560461,52.079971804],[4.325154902,52.081090447],[4.324341193,52.082813095],[4.324253991,52.082905802],[4.326214204,52.083908049],[4.327638547,52.085278887],[4.331766125,52.086890801],[4.334888355,52.088443843],[4.337174602,52.089716008],[4.342760743,52.09161594]],[[4.330996337,52.083468434],[4.331154279,52.083552339],[4.331260659,52.083457059],[4.331993065,52.083782098],[4.331971994,52.083887302],[4.332085912,52.083945598],[4.332221278,52.083882658],[4.332931501,52.084202909],[4.332892698,52.084271298],[4.333106468,52.084353407],[4.333321609,52.084220151],[4.333626928,52.084362637],[4.3336211,52.084610035],[4.333974603,52.084759836],[4.334098339,52.08471741],[4.334706094,52.084967997],[4.334677859,52.085061686],[4.334839897,52.085129578],[4.334934104,52.085077718],[4.335461192,52.085279465],[4.335406434,52.085393537],[4.335583045,52.085475301],[4.33568872,52.0854098],[4.336222825,52.085629943],[4.336179767,52.085721207],[4.336315838,52.085786573],[4.336443294,52.085744177],[4.33692146,52.085970685],[4.33696361,52.086076453],[4.337349748,52.086248621],[4.337487287,52.086234817],[4.337627655,52.086296939],[4.337583792,52.086341159],[4.337676738,52.086378427],[4.337731731,52.08633871],[4.338013443,52.08647177],[4.337963266,52.08655437],[4.338079214,52.086611321],[4.338162821,52.086595193],[4.338576951,52.086779309],[4.33855633,52.086836087],[4.338687436,52.086898817],[4.338753595,52.086859523],[4.339143353,52.08704031],[4.339193589,52.087164279],[4.339107286,52.087239403],[4.338883296,52.087210624],[4.337206121,52.086428526],[4.337191749,52.086383751],[4.336796885,52.086207856],[4.336599316,52.086226715],[4.336117806,52.08598414],[4.336137791,52.08592475],[4.336038527,52.085875748],[4.335907889,52.085895203],[4.333900626,52.084997241],[4.333891398,52.084883514],[4.333523144,52.084728088],[4.333350099,52.084748536],[4.332908618,52.084544795],[4.332964573,52.084442654],[4.332793602,52.084375138],[4.332633536,52.084476348],[4.331970994,52.084184477],[4.331963839,52.084109268],[4.331772127,52.084037898],[4.331620347,52.084040205],[4.331025583,52.083776433],[4.331076845,52.083621091],[4.330932864,52.083544646],[4.330732016,52.083608832],[4.329846737,52.083426762],[4.329643836,52.083325967],[4.328903665,52.082543984],[4.328969088,52.082384842],[4.329348871,52.08236534],[4.331128227,52.083238211],[4.330996337,52.083468434]],[[4.342735901,52.084422865],[4.342759811,52.084570834],[4.34223592,52.084594225],[4.342233757,52.084435487],[4.342735901,52.084422865]]]}}}'

After the first request, I parse the next page url indicated in the response:
_links": {
“next”: {
“href”: "https://bag.basisregistraties.overheid.nl/api/v1/verblijfsobjecten?pageSize=100&page=2

Then request again and again in a recursive loop, until there is no more “next” links.

But for this geometry in the body, there are 7500 “Nummeraanduidingen” objects, so which means I have to make the request 75 times, every time 100 objects (param: pageSize=100). But this request fails every single time, around the 50th time. And each call takes about 30-40 seconds, is this true?

Thank you very much for the reply in advance. If you need my complete code (in JAVA), please let me know and I will add it,

I’ve been able to reproduce the issue. Your request are correct and should give results.

We’ve checked for obvious errors in our backend and haven’t found any. We’ll look further into it, but this may take a while (days, not months).

I have not been able to reproduce the issue with a smaller search area (polygon). Splitting your search area may be a temporary workaround.

Thanks Robin, you are right about the smaller search area works fine for me also. I will see if I can split the coordinates for now.

Looking forward to your updates on this.

Hi Robin,

Another error I got when fetching a lot of data (this time only 800 Nummeraanduidingen objects, but i need to send request 800 times also), then I got this:

Account plan limit exceeded

Reason: Not Set|270b54bb-8146-49b1-91ef-0dcf74b7f72b|5d5d0165-3022-4497-b396-d68b95191a50|quota?failed"

Any idea why is this happen? Do I need to pay for the service to remove limit?

There is an undocumented rate limit on the API to protect the backend. It was 30 requests/second and has now been increased to 100/sec.

Strictly speaking the BAG REST API is not a download service and hence not meant for requesting huge amounts of objects.

1 like

The error message for “Too many requests” has been updated. You’d now get a problem+json response with a clear reason.

Regarding the issue with the 500s, we are now testing a fix and expect to update the API in production some time next week.

2 likes