SpiceInit with web support fails

[tckb]

ISIS version(s) affected: 8.0.2

Description
Yesterday 09.03.2024 LRO released new batch of data products LROC 57C. I tried the calibration as usual but there was this error from spiceinit

How to reproduce

Download M1447547309LC

lronac2isis from =  M1447547309LC.IMG  to = M1447547309LC.cub
spiceinit from = M1447547309LE.cub web=yes

**ERROR** The Spice Server was unable to initialize the cube.  The error reported was: No Camera Kernel found for the image [/tmp/spice_web_service/V0pkx7cROS].

Possible Solution
It's possible that this is related #5113 which suggests that WebSPICE needs to be updated.

[tckb]

@rvwagner I referred to your issue - I see the same issue as you pointed earlier. Do you have the same issue with 57C?

cc: @Kelvinrr

[rvwagner]

Yep, I can confirm that that image isn't working with web SPICE, but is working with the current local SPICE directory. Sounds like the USGS needs to update their web spice server again, and should probably make a monthly task to check on it, since LROC data is released monthly.

[tckb]

okay good to have it validated!! I've only recently started looking at ISIS3 cub files and oh my I never really bothered about looking at local but - this is making me go crazy! How big is LRO data anyways @rvwagner ? :D I have 400 Gib synced and it says more 300Gib and filtering doesn't seem to work. I managed to download the rest directly using rclone

[rvwagner]

You might need to double-check that you're only downloading the LRO SPICE data, it looks like it should only be ~286 GB. It is really big, though (5-10 data points per second for 15 years is a lot of data).

[tckb]

I would have imagined it would be about 300Gi.

I used the following to sync the data.

downloadIsisData lro $ISISDATA

I am not sure how else to download just the lro data. I'll just use rclone directly -

[tckb]

Also related there seems to be a new spice webserver: #5327

[Kelvinrr]

Did changing the spice server fix things or is it an issue with both? The spiceserver path is going to be updated with the next release of ISIS so for now you still need to change it via the URL param.

[rvwagner]

Adding url=https://astrogeology.usgs.gov/apis/ale/v0.9.1/spiceserver/ fixes the problem for me. When is the old server going to be taken offline, and would you be able to keep it updated with new kernels until then to support people who aren't on the latest not-yet-released version of ISIS, and aren't checking this GitHub page?

Also, is there/can there be an IsisPreferences entry for "default web SPICE" URL?

[Kelvinrr]

The old one will stay online as long as it's default in older versions of ISIS. At some point we will need to backport the change.

[Kelvinrr]

#5431 was merged just now, if we dont backport (as it would take a lot of time), we will re-route the old URL to the new one so users on old versions will be none the wiser to this whole situation.

[tckb]

thanks @Kelvinrr ! I just tested it and it works - its routing to the old webserver. Technically this issue is fixed. But the bigger question is when and how is web server is updated? I guess I want to know if web will be in sync with PDS data release schedules.

This time I ended up using the local instead of web because its of the quick availability of my data server and is not portable solution.

FYI logs of spiceinit with web

(....)

  Group = UserParameters
    FROM         = M1410024427RE.cub
    WEB          = true
    ATTACH       = TRUE
    CKSMITHED    = FALSE
    CKRECON      = TRUE
    CKPREDICTED  = FALSE
    CKNADIR      = FALSE
    SPKSMITHED   = true
    SPKRECON     = TRUE
    SPKPREDICTED = FALSE
    SHAPE        = SYSTEM
    STARTPAD     = 0.0
    ENDPAD       = 0.0
    URL          = https://services.isis.astrogeology.usgs.gov/cgi-bin/spicei-
                   nit.cgi
    PORT         = 443
  End_Group

  Group = Kernels
    NaifFrameCode             = -85610
    LeapSecond                = $base/kernels/lsk/naif0012.tls
    TargetAttitudeShape       = ($base/kernels/pck/pck00009.tpc,
                                 $lro/kernels/pck/moon_080317.tf,
                                 $lro/kernels/pck/moon_assoc_me.tf)
    TargetPosition            = (Table,
                                 $lro/kernels/tspk/moon_pa_de421_1900-2050.bpc,
                                 $lro/kernels/tspk/de421.bsp)
    InstrumentPointing        = (Table,
                                 $lro/kernels/ck/lrolc_2022151_2022182_v01.bc,
                                 $lro/kernels/ck/moc42r_2022151_2022182_v01.bc,
                                 $lro/kernels/fk/lro_frames_2014049_v01.tf)
    Instrument                = $lro/kernels/ik/lro_lroc_v18.ti
    SpacecraftClock           = $lro/kernels/sclk/lro_clkcor_2023103_v00.tsc
    InstrumentPosition        = (Table,
                                 $lro/kernels/spk/LRO_ES_115_202211_GRGM900C_L-
                                 600.BSP)
    InstrumentAddendum        = $lro/kernels/iak/lro_instrumentAddendum_v05.ti
    ShapeModel                = $base/dems/ldem_128ppd_Mar2011_clon180_radius-
                                _pad.cub
    InstrumentPositionQuality = Smithed
    InstrumentPointingQuality = Reconstructed
    CameraVersion             = 2
  End_Group

  Group = Accounting
    ConnectTime = 00:00:06.2
    CpuTime     = 00:00:06.0
  End_Group
End_Object
End```

[rvwagner]

The kernels that are downloaded when you use local kernels are the same ones we use internally, and include an extra ~3 months past the PDS-released images. If the USGS can update their web SPICE server at least once every month or two, then all the publicly released images will be supported.

Anyway, it seems like putting a redirect in on the old URL should solve this problem (assuming the new server is reliably updating), but until that is put in, can you guys maybe do a manual update of the kernels on the old server so that it will at least support all the data that is currently released to the PDS? Resyncing now should buy you until June 15th to put in the redirect before new public LROC data stops working again.

[Kelvinrr]

@tckb so for both servers the kernels are updated every other day. For the AWS backed one, it updated via runner on a schedule into an elastic file system. It's very low maintenance.

The old one is via a cronjob. Our IT manages the old one and since it's a physical machine it tends to run out of space among other issues. All of which they have to deal with on top of their other responsibilities. Hence why it tends to go down and why we are retiring it.

[tckb]

spice data of LRO for local syncing uses rclone backed by AWS

[asc_s3]
type = s3
provider = AWS
region = us-west-2
location_constraint = us-west-2

[lro]
type = alias
remote = asc_s3:asc-isisdata/usgs_data/lro/

@rvwagner are you using the same server?

@Kelvinrr I understand - I think you mentioned it before about the disk issues elsewhere. So the new v0.9.1 version is back up by this aws?

[Kelvinrr]

So the new v0.9.1 version is back up by this aws?

@tckb I need you to clarify what you mean by this.

[tckb]

ah, sorry I meant by v0.91 i mean this webserver https://astrogeology.usgs.gov/apis/ale/v0.9.1/spiceserver/ - does this server use asc_s3 mentioned above. By that I want to understand if we are using the same server in local and web mode

[Kelvinrr]

both https://astrogeology.usgs.gov/apis/ale/v0.9.1/spiceserver/ and the old default URL use the same spice server app. v0.9.1 also use the same ISIS Data that you'd get with downloadIsisData, the main difference is the old default URL sometimes has storage issues. Hopefully that answers your question.

[tckb]

Yes, thank you @Kelvinrr that is enough. I'll close this