Blob coming back after few days deleting!

[a45hif]

Blob is not deleting permanently for dictionaries data type. For some days after deleting it says "Blob does not exists" but later after few more days it starts existing with all the data which was saved before deleting plus adding 2 more keys in the dict at top (attached in img). Make deletion permanent for dictionaries data type. This is not happening with lists data type.

[tburch]

When you make a delete, it's removed from the file system. There's no process in place that can recover that.

[a45hif]

But it is returning 🥴 I don't know how.
Even after deleting it via JsonBlob API, it recovers with all data present before deletion automatically after few days and I'm able to edit/add data in that again. I'd noticed this strange behaviour many times with different blob also. Maybe any kind of rollback or something 🧐
I've a blob which created a year ago and after few months of inactivity it was deleted automatically by the system and I was not able to access the blob via browser but as I didn't changed in my application so app was requesting that blob daily twice for backup. After few days of continued request it came back with all the data. This happened to another blob also recently which I'd deleted few weeks ago but is now available to use with all data. Verify it by yourself🥴
I'm also verifying now. I'll let you updated if it happens again.

[tburch]

Here's a blob I just created and verified it returns a 404 when requesting it. I'll keep and eye on it, but I don't see a way that it can be restored...

~> curl -i -X "POST" -d '{"people":["bill", "steve", "bob"]}' -H "Content-Type: application/json" -H "Accept: application/json" https://jsonblob.com/api/jsonBlob
HTTP/2 201 
date: Mon, 04 Jul 2022 14:41:46 GMT
content-type: application/json
content-length: 35
location: http://jsonblob.com/api/jsonBlob/993527046336692224
cf-cache-status: DYNAMIC
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=jSDp3pz5aSrJtBNirXaJGolT1ToMBT%2FQiMKPmF%2FX64YsylHUqguhTw0cX%2BpNAVZaeOE56br0OFz7ctIJPF1Pt5zH34B8bmo3GpbHKaA5gzpFvjbA%2Fg9CVHMb53gUCLg%3D"}],"group":"cf-nel","max_age":604800}
nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
server: cloudflare
cf-ray: 72589faa5fc02f79-DFW
alt-svc: h3=":443"; ma=86400, h3-29=":443"; ma=86400

{"people":["bill", "steve", "bob"]}⏎                                                                                                        

~> curl -i -H "Content-Type: application/json" -H "Accept: application/json" https://jsonblob.com/api/jsonBlob/993527046336692224
HTTP/2 200 
date: Mon, 04 Jul 2022 14:42:02 GMT
content-type: application/json
content-length: 35
cf-cache-status: DYNAMIC
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=YMh7KR9UxShjQ%2FeYWeXQuafkaJAtXY%2Fqq3hrpx7ZSNW%2Bv5XN%2F92UhBW29cIpAMNubUSNGGE0uA6gKZoI98LY0Dr%2BDZXGjDU0jUXwkr2X3VVFmin1fgaS%2F%2FU%2BZ1sNxmE%3D"}],"group":"cf-nel","max_age":604800}
nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
server: cloudflare
cf-ray: 7258a00c8db5286f-DFW
alt-svc: h3=":443"; ma=86400, h3-29=":443"; ma=86400

{"people":["bill", "steve", "bob"]}⏎                                                                                                         t

~> curl -i -X "DELETE" https://jsonblob.com/api/jsonBlob/993527046336692224
HTTP/2 200 
date: Mon, 04 Jul 2022 14:42:28 GMT
cf-cache-status: DYNAMIC
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=fnhYPYHZSQn2ZUSO2EWQq2LlH6rubvNnuhD%2Fy2RKDj2KK2XbVUQxycgmw%2F6OHyHg%2Fr%2BOoXxAzvvy8tqxZ9HtrOqijOhjsmuAUWCtP6GLVqYNXbbT9BzAx%2Bm4UB0LK9Q%3D"}],"group":"cf-nel","max_age":604800}
nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
server: cloudflare
cf-ray: 7258a0b00d01a9f7-DFW
alt-svc: h3=":443"; ma=86400, h3-29=":443"; ma=86400

~> curl -i -H "Content-Type: application/json" -H "Accept: application/json" https://jsonblob.com/api/jsonBlob/993527046336692224
HTTP/2 404 
date: Mon, 04 Jul 2022 14:42:35 GMT
content-type: application/json
content-length: 140
cf-cache-status: DYNAMIC
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=zDJxt6FIufRHU%2F2wrRxfdFjnuM1qD3khRxjU%2B4TKIbS8pdBhayzcnK4y%2BpJqbiLc7njQwuqe0mDtZVmNwzhuw4IuGem4i73b%2BtcbrQPkzBBOyesae%2B5JrHXoFA3Hn%2Bw%3D"}],"group":"cf-nel","max_age":604800}
nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
server: cloudflare
cf-ray: 7258a0d85e782893-DFW
alt-svc: h3=":443"; ma=86400, h3-29=":443"; ma=86400

{"message":"Blob with id 993527046336692224 does not exist","_links":{"self":{"href":"/api/jsonBlob/993527046336692224","templated":false}}}⏎

[a45hif]

Okay I'm too checking! Meanwhile here's a blob id which got restored 904641375539314688. Keys at index 1 & 2 (now) were at the time of deletion. As I deleted the blob i was getting same response as you stated above but after few days all the data came back. Maybe you can check when it was deleted and when it came back from logs.

[tburch]

Hi @uraashif. I don't see anything abnormal in the logs for 904641375539314688, but I also don't log all of the requests for the blobs.

[a45hif]

Maybe the issue is that we can still update the data of blob which is deleted in the past. I just added a key in your 993527046336692224 blob and it was successful. If blob does not exists then how can anyone update it? 😵

{ "message": "Blob with id 993527046336692224 does not exist", "_links": { "self": { "href": "/api/jsonBlob/993527046336692224", "templated": false } }, "name":"bob" }

[tburch]

@uraashif I think you might be correct. There's no check to make sure a blob exists when a PUT is made against the blob id.

 ~  curl -i -X "DELETE" https://jsonblob.com/api/jsonBlob/993527046336692224                                                  263ms  Thu Jul  7 11:28:52 2022
HTTP/2 200 
date: Thu, 07 Jul 2022 17:28:54 GMT
cf-cache-status: DYNAMIC
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=1OkIA1F64a0%2BrG3n%2FM7kBgsT22cW7eoWR8GeRJkWQtghKMol34lv5a7Qawpc2TZu3BgeBqInHFWYXaXEb66t1FmjBZNEe6izSlABkEiPIbEJ2AD6f6c6GqtXiQxmyls%3D"}],"group":"cf-nel","max_age":604800}
nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
server: cloudflare
cf-ray: 72724c9d9edbf222-ORD
alt-svc: h3=":443"; ma=86400, h3-29=":443"; ma=86400

~  curl -i -H "Content-Type: application/json" -H "Accept: application/json" 'https://jsonblob.com/api/jsonBlob/993527046336692224'
HTTP/2 404 
date: Thu, 07 Jul 2022 17:29:49 GMT
content-type: application/json
content-length: 140
cf-cache-status: DYNAMIC
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=LYE%2BNnI%2F65uF8spxXaLySJNJOCTwNtc0tG7b4i2hEPj1yaVea%2Bvo5bDerQ%2Foy1Ro2GGTQg87yNqUbHSyZ13FPkZqhVlYjjwruWLXxOmP2PYcmuMtkf9C5dtRwBUu4II%3D"}],"group":"cf-nel","max_age":604800}
nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
server: cloudflare
cf-ray: 72724df22c24f222-ORD
alt-svc: h3=":443"; ma=86400, h3-29=":443"; ma=86400

{"message":"Blob with id 993527046336692224 does not exist","_links":{"self":{"href":"/api/jsonBlob/993527046336692224","templated":false}}}⏎                   

~  curl -i -X "PUT" -d '{"people":["fred", "mark", "andrew"]}' -H "Content-Type: application/json" -H "Accept: application/json" https://jsonblob.com/api/jso
nBlob/993527046336692224
HTTP/2 200 
date: Thu, 07 Jul 2022 17:30:05 GMT
content-type: application/json
content-length: 37
cf-cache-status: DYNAMIC
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=CLJW3w8eS2RAXw1VHo6JiOk496fdn2UHwu8ER5j1fWK08mbIS8FRTKCnV%2FGVKyY3HR9BkTRQIezJ92Os%2Fa0JjJx0JiCt3sO9xIoGSH50sngeau4EOeWRyV0uemBE9uA%3D"}],"group":"cf-nel","max_age":604800}
nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
server: cloudflare
cf-ray: 72724e55ea36f222-ORD
alt-svc: h3=":443"; ma=86400, h3-29=":443"; ma=86400

{"people":["fred", "mark", "andrew"]}⏎                                                                                                                          

~  curl -i -H "Content-Type: application/json" -H "Accept: application/json" 'https://jsonblob.com/api/jsonBlob/993527046336692224'
HTTP/2 200 
date: Thu, 07 Jul 2022 17:30:09 GMT
content-type: application/json
content-length: 37
cf-cache-status: DYNAMIC
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=Yyda%2BncrGTCnXojNn%2FooEAaaBgmWwa3WeTCiM3W3qbzKf8QfgXuZ8fM9jo%2F%2FF3z5EEqZZ9wd0lMNg9dG3AG8vlPS3AwuJ7jhaPcVuyYfBFNcL3NF3v8MgPeah5sWm98%3D"}],"group":"cf-nel","max_age":604800}
nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
server: cloudflare
cf-ray: 72724e72186bf222-ORD
alt-svc: h3=":443"; ma=86400, h3-29=":443"; ma=86400

{"people":["fred", "mark", "andrew"]}⏎ ```

[tburch]

Thanks, @uraashif! I'll deploy the version with the fix tonight.

[a45hif]

Okay!👍