How to batch update missing blockindex ?

[longvic0]

Hi all,
I've got a problem while deploy an explorer site for a dead coin, they need to build for swap process.
My explorer's updated to the latest block. But I find some missing blocks between block #1 and latest blocks ( about 8000 blockindex doesn't exist in my explorer database , Just export all the blockindex from db: mongoexport -d databasename -c txes -f blockindex --query '{ blockindex:{$gte:0}}' -o /root/all-blocks.json , then listed all missing blocks ).
The blockindex will be updated one by one when I find that blockindex from search function.
Is there any solution to batch update missing blockindex

Thanks.

[uaktags]

You could edit either the Benchmark or the Sync to modify the start and stop areas to a hard coded blockindices you need.

For example (not tested code) say that you need blocks 8000 until 8888

explorer/scripts/sync.js

Lines 171 to 178 in 30f91a0

	} else if (mode == 'check') {
	db.update_tx_db(settings.coin, 1, stats.count, settings.check_timeout, function(){
	db.get_stats(settings.coin, function(nstats){
	console.log('check complete (block: %s)', nstats.last);
	exit();
	});
	});
	} else if (mode == 'update') {

modify to something like

} else if (mode == 'check') {
                    db.update_tx_db(settings.coin, 8000, 8888, settings.check_timeout, function(){
                      db.get_stats(settings.coin, function(nstats){
                        console.log('check complete (block: %s)', nstats.last);
                        exit();
                      });
                    });
                  }

Modify as needed.

Keep in mind what coinstats.last and coinstats.count shows in mongo, as I think these will get modified by the sync.js during its running.

[longvic0]

Thank you @uaktags ,
After make change this, I'll run with check mode , is it right ?
node scripts/sync.js check

My temp solution is :

1. get blockhash via blockindex : explorer/
   http://explorersite.com/api/getblockhash?index=blockindex
2. get update for that blockhash
   http://explorersite.com/block/blockhash
3. run it.

<iframe src='http://explorersite.com/block/blockhash' width=0 height=0 ></iframe> But this take too much time to finish 8000 or 80K blocks . ( I'm not a coder :D )

[uaktags]

check mode is the following
node scripts/sync.js index check

Make sure you make a mongo backup first just to be safe (of course).

[uaktags]

Another idea may be to use bash:

λ cat test.sh
#!/bin/bash
for i in {8000..8888}
do
   curl "http://127.0.0.1:3001/block/$1"
done

so that you can run it with

λ ./test.sh

That will do a curl out to your explorer (presuming 127.0.0.1:3001) from block 8000 to 8888. It's a stupid script, and i'd recommend it in a separate ssh window or a separate "screen" because I'm just too lazy to look up curl's output redirect to hide all the html sourcecode that'll be returned.

[longvic0]

I think the bash can't run because the url: /block/ only accept the blockhash.
I'll try to modify the sync.js.
Thank you , @uaktags

[uaktags]

hmm, /block/:number should work just fine, atleast it does for me, but if i'm being honest, i have ran a vanilla Iquidus in a long time so perhaps i am mistaken.

Let me know how the sync.js works out, and if that fails for whatever reason, I'll look into typing something up more thoroughly for you.

[longvic0]

Hi,
The sync.js in check mode works well for range of missing blocks. And I just run the script in background process. It doesn't create an index.pid file. Thank you, you save my day :)

node scripts/sync.js index check &

[longvic0]

Hi Tim,
I've finished node scripts/sync.js index check process. And I rollback sync.js file & updated the coinstats.last value in coinstats to last blockindex of txes.blockindex collection already.
But I got problem while run node scripts/sync.js index update process:
Coinstats.last not updated:
{ "_id" : ObjectId("5cbc2ac32931cac22c6c6795"), "coin" : "Coinname", "last_price" : 0, "connections" : 4, "supply" : 60014874.99999456, "last" : 106279, "count" : 1270634, "__v" : 0 }
Temporary method
db.coinstats.update{"last" : 101279 },{"last" : 1269738});
Then run index update. When it finish index update process, it return not updated coinstats.last.
Please give me some advice to this issue.
Thanks