error: found unstable fingerprints for evaluate_obligation #83291

aurexav · 2021-03-19T05:15:55Z

Code

Error output

xavier on xavier.zone in common at  rococo-v1 <!?> via 𝓡 v1.52.0-nightly takes 13ms
Fri Mar 19 13:12:30 2021 - 45%
λ cargo b --release                     
   Compiling darwinia-ethereum-relay v2.0.0 (/home/xavier/Documents/darwinia-network/common/frame/bridge/ethereum/relay)
   Compiling darwinia-ethereum-linear-relay v2.0.0 (/home/xavier/Documents/darwinia-network/common/frame/bridge/ethereum/linear-relay)
   Compiling pangolin-runtime v2.0.0 (/home/xavier/Documents/darwinia-network/common/bin/node/runtime/pangolin)
thread 'rustc' panicked at 'found unstable fingerprints for evaluate_obligation(1746fd5d1304336-c1d2d55e63e10b17)', /rustc/1705a7d64b833d1c4b69958b0627bd054e6d764b/compiler/rustc_query_system/src/query/plumbing.rs:593:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/issues/new?labels=C-bug%2C+I-ICE%2C+T-compiler&template=ice.md

note: rustc 1.52.0-nightly (1705a7d64 2021-03-18) running on x86_64-unknown-linux-gnu

note: compiler flags: -C opt-level=3 -C embed-bitcode=no -C incremental --crate-type lib

note: some of the compiler flags provided by cargo are hidden

query stack during panic:
#0 [evaluate_obligation] evaluating trait selection obligation `impls::DealWithFees: sp_api_hidden_includes_construct_runtime::hidden_include::traits::OnUnbalanced<darwinia_balances::NegativeImbalance<Runtime, darwinia_balances::Instance0>>`
#1 [normalize_projection_ty] normalizing `Canonical { max_universe: U0, variables: [], value: ParamEnvAnd { param_env: ParamEnv { caller_bounds: [], reveal: All }, value: ProjectionTy { substs: [pallet_transaction_payment::CurrencyAdapter<darwinia_balances::Module<Runtime, darwinia_balances::Instance0>, impls::DealWithFees>, Runtime], item_def_id: DefId(394:165 ~ pallet_transaction_payment[3044]::payment::OnChargeTransaction::Balance) } } }`
end of query stack
error: could not compile `pangolin-runtime`

To learn more, run the command again with --verbose.

Backtrace

xavier on xavier.zone in common at  rococo-v1 <!?> via 𝓡 v1.52.0-nightly takes 62ms
Fri Mar 19 13:14:24 2021 - 46%
λ RUST_BACKTRACE=1 cargo b --release 
   Compiling librocksdb-sys v6.11.4
   Compiling sc-network v0.9.0 (https://github.com/darwinia-network/substrate.git?tag=rococo-v1.1#645299e8)
   Compiling pangolin-runtime v2.0.0 (/home/xavier/Documents/darwinia-network/common/bin/node/runtime/pangolin)
thread 'rustc' panicked at 'found unstable fingerprints for evaluate_obligation(1746fd5d1304336-c1d2d55e63e10b17)', /rustc/1705a7d64b833d1c4b69958b0627bd054e6d764b/compiler/rustc_query_system/src/query/plumbing.rs:593:5
stack backtrace:
   0: rust_begin_unwind
             at /rustc/1705a7d64b833d1c4b69958b0627bd054e6d764b/library/std/src/panicking.rs:493:5
   1: std::panicking::begin_panic_fmt
             at /rustc/1705a7d64b833d1c4b69958b0627bd054e6d764b/library/std/src/panicking.rs:435:5
   2: rustc_query_system::query::plumbing::incremental_verify_ich
   3: rustc_query_system::query::plumbing::load_from_disk_and_cache_in_memory
   4: rustc_data_structures::stack::ensure_sufficient_stack
   5: rustc_query_system::query::plumbing::get_query_impl
   6: <rustc_query_impl::Queries as rustc_middle::ty::query::QueryEngine>::evaluate_obligation
   7: <rustc_infer::infer::InferCtxt as rustc_trait_selection::traits::query::evaluate_obligation::InferCtxtExt>::evaluate_obligation
   8: <rustc_infer::infer::InferCtxt as rustc_trait_selection::traits::query::evaluate_obligation::InferCtxtExt>::evaluate_obligation_no_overflow
   9: rustc_trait_selection::traits::fulfill::FulfillProcessor::process_trait_obligation
  10: rustc_trait_selection::traits::fulfill::FulfillProcessor::progress_changed_obligations
  11: rustc_data_structures::obligation_forest::ObligationForest<O>::process_obligations
  12: <rustc_trait_selection::traits::fulfill::FulfillmentContext as rustc_infer::traits::engine::TraitEngine>::select_where_possible
  13: <rustc_infer::infer::InferCtxtBuilder as rustc_trait_selection::infer::InferCtxtBuilderExt>::enter_canonical_trait_query
  14: rustc_traits::normalize_projection_ty::normalize_projection_ty
  15: rustc_query_impl::<impl rustc_query_system::query::config::QueryAccessors<rustc_query_impl::plumbing::QueryCtxt> for rustc_query_impl::queries::normalize_projection_ty>::compute
  16: rustc_query_system::query::config::QueryVtable<CTX,K,V>::compute
  17: rustc_middle::dep_graph::<impl rustc_query_system::dep_graph::DepKind for rustc_middle::dep_graph::dep_node::DepKind>::with_deps
  18: rustc_query_system::dep_graph::graph::DepGraph<K>::with_ignore
  19: rustc_query_system::query::plumbing::load_from_disk_and_cache_in_memory
  20: rustc_data_structures::stack::ensure_sufficient_stack
  21: rustc_query_system::query::plumbing::get_query_impl
  22: <rustc_query_impl::Queries as rustc_middle::ty::query::QueryEngine>::normalize_projection_ty
  23: <rustc_trait_selection::traits::query::normalize::QueryNormalizer as rustc_middle::ty::fold::TypeFolder>::fold_ty
  24: rustc_middle::ty::fold::TypeFoldable::fold_with
  25: rustc_middle::ty::structural_impls::<impl rustc_middle::ty::fold::TypeFoldable for &rustc_middle::ty::TyS>::super_fold_with
  26: <rustc_trait_selection::traits::query::normalize::QueryNormalizer as rustc_middle::ty::fold::TypeFolder>::fold_ty
  27: rustc_middle::ty::fold::TypeFoldable::fold_with
  28: rustc_middle::ty::structural_impls::<impl rustc_middle::ty::fold::TypeFoldable for &rustc_middle::ty::TyS>::super_fold_with
  29: <rustc_trait_selection::traits::query::normalize::QueryNormalizer as rustc_middle::ty::fold::TypeFolder>::fold_ty
  30: <rustc_infer::infer::at::At as rustc_trait_selection::traits::query::normalize::AtExt>::normalize
  31: rustc_infer::infer::InferCtxtBuilder::enter
  32: rustc_traits::normalize_erasing_regions::normalize_generic_arg_after_erasing_regions
  33: rustc_middle::dep_graph::<impl rustc_query_system::dep_graph::DepKind for rustc_middle::dep_graph::dep_node::DepKind>::with_deps
  34: rustc_query_system::dep_graph::graph::DepGraph<K>::with_ignore
  35: rustc_query_system::query::plumbing::load_from_disk_and_cache_in_memory
  36: rustc_data_structures::stack::ensure_sufficient_stack
  37: rustc_query_system::query::plumbing::get_query_impl
  38: <rustc_query_impl::Queries as rustc_middle::ty::query::QueryEngine>::normalize_generic_arg_after_erasing_regions
  39: <rustc_middle::ty::normalize_erasing_regions::NormalizeAfterErasingRegionsFolder as rustc_middle::ty::fold::TypeFolder>::fold_ty
  40: rustc_middle::ty::instance::Instance::subst_mir_and_normalize_erasing_regions
  41: <rustc_mir::monomorphize::collector::MirNeighborCollector as rustc_middle::mir::visit::Visitor>::visit_terminator
  42: rustc_mir::monomorphize::collector::collect_neighbours
  43: rustc_mir::monomorphize::collector::collect_items_rec
  44: rustc_mir::monomorphize::collector::collect_items_rec
  45: rustc_mir::monomorphize::collector::collect_items_rec
  46: rustc_mir::monomorphize::collector::collect_items_rec
  47: rustc_mir::monomorphize::collector::collect_items_rec
  48: rustc_mir::monomorphize::collector::collect_items_rec
  49: rustc_mir::monomorphize::collector::collect_items_rec
  50: rustc_mir::monomorphize::collector::collect_items_rec
  51: rustc_mir::monomorphize::collector::collect_items_rec
  52: rustc_mir::monomorphize::collector::collect_items_rec
  53: rustc_mir::monomorphize::collector::collect_items_rec
  54: rustc_mir::monomorphize::collector::collect_items_rec
  55: rustc_mir::monomorphize::collector::collect_crate_mono_items
  56: rustc_mir::monomorphize::partitioning::collect_and_partition_mono_items
  57: rustc_query_impl::<impl rustc_query_system::query::config::QueryAccessors<rustc_query_impl::plumbing::QueryCtxt> for rustc_query_impl::queries::collect_and_partition_mono_items>::compute
  58: rustc_middle::dep_graph::<impl rustc_query_system::dep_graph::DepKind for rustc_middle::dep_graph::dep_node::DepKind>::with_deps
  59: rustc_query_system::dep_graph::graph::DepGraph<K>::with_task_impl
  60: rustc_data_structures::stack::ensure_sufficient_stack
  61: rustc_query_system::query::plumbing::force_query_with_job
  62: rustc_query_system::query::plumbing::get_query_impl
  63: <rustc_query_impl::Queries as rustc_middle::ty::query::QueryEngine>::collect_and_partition_mono_items
  64: <rustc_codegen_llvm::LlvmCodegenBackend as rustc_codegen_ssa::traits::backend::CodegenBackend>::codegen_crate
  65: rustc_interface::passes::QueryContext::enter
  66: rustc_interface::queries::Queries::ongoing_codegen
  67: rustc_interface::queries::<impl rustc_interface::interface::Compiler>::enter
  68: rustc_span::with_source_map
  69: rustc_interface::interface::create_compiler_and_run
  70: scoped_tls::ScopedKey<T>::set
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/issues/new?labels=C-bug%2C+I-ICE%2C+T-compiler&template=ice.md

note: rustc 1.52.0-nightly (1705a7d64 2021-03-18) running on x86_64-unknown-linux-gnu

note: compiler flags: -C opt-level=3 -C embed-bitcode=no -C incremental --crate-type lib

note: some of the compiler flags provided by cargo are hidden

query stack during panic:
#0 [evaluate_obligation] evaluating trait selection obligation `impls::DealWithFees: sp_api_hidden_includes_construct_runtime::hidden_include::traits::OnUnbalanced<darwinia_balances::NegativeImbalance<Runtime, darwinia_balances::Instance0>>`
#1 [normalize_projection_ty] normalizing `Canonical { max_universe: U0, variables: [], value: ParamEnvAnd { param_env: ParamEnv { caller_bounds: [], reveal: All }, value: ProjectionTy { substs: [pallet_transaction_payment::CurrencyAdapter<darwinia_balances::Module<Runtime, darwinia_balances::Instance0>, impls::DealWithFees>, Runtime], item_def_id: DefId(394:165 ~ pallet_transaction_payment[3044]::payment::OnChargeTransaction::Balance) } } }`
#2 [normalize_generic_arg_after_erasing_regions] normalizing `for<'r> fn(&'r mut _::_parity_scale_codec::depth_limit::DepthTrackingInput<&[u8]>) -> std::result::Result<<<<Runtime as pallet_transaction_payment::Config>::OnChargeTransaction as pallet_transaction_payment::OnChargeTransaction<Runtime>>::Balance as _::_parity_scale_codec::HasCompact>::Type, _::_parity_scale_codec::Error> {<<<<Runtime as pallet_transaction_payment::Config>::OnChargeTransaction as pallet_transaction_payment::OnChargeTransaction<Runtime>>::Balance as _::_parity_scale_codec::HasCompact>::Type as _::_parity_scale_codec::Decode>::decode::<_::_parity_scale_codec::depth_limit::DepthTrackingInput<&[u8]>>}`
#3 [collect_and_partition_mono_items] collect_and_partition_mono_items
end of query stack
error: could not compile `pangolin-runtime`

To learn more, run the command again with --verbose.

The text was updated successfully, but these errors were encountered:

jyn514 · 2021-03-19T09:45:16Z

Could you post the code this crashes on? It doesn't have to be small, but this is hard to fix without a way to replicate the crash.

aurexav · 2021-03-19T10:46:14Z

Could you post the code this crashes on? It doesn't have to be small, but this is hard to fix without a way to replicate the crash.

I'm not sure what causes this. Can't give a minimal env to reproduce this, sorry.

I'm compiling a substrate runtime. Really big project.

darwinia-network/darwinia-common@81fb0c5

The first time it can be compile success. And I do some modifies to code, then it panics.

And cargo clean is a workaround.

Aaron1011 · 2021-03-19T17:18:42Z

This will be fixed by #83220

aurexav · 2021-03-25T09:33:29Z

Fixed in 3-23

But in 3-24 panic again

teor2345 · 2021-03-27T05:15:31Z

I'm also seeing a similar error intermittently, when compiling https://github.com/ZcashFoundation/zebra using clippy --all-targets --all-features. See #83381 for more details.

It seems like nightly-2021-03-13-x86_64-unknown-linux-gnu is unaffected (or the issue is very rare).

2021-03-14 to 2021-03-21 are affected occasionally. (Maybe 1 in 10 runs.)

I haven't tested anything more recent, but I'll do that over the next few days.

aurexav · 2021-03-27T14:53:52Z

I'm also seeing a similar error intermittently, when compiling https://github.com/ZcashFoundation/zebra using clippy --all-targets --all-features. See #83381 for more details.

It seems like nightly-2021-03-13-x86_64-unknown-linux-gnu is unaffected (or the issue is very rare).

2021-03-14 to 2021-03-21 are affected occasionally. (Maybe 1 in 10 runs.)

I haven't tested anything more recent, but I'll do that over the next few days.

Now I stay at 03-01

Quite stable

real-felix · 2021-03-31T18:10:58Z

I'm on 2021-03-24, and I'm getting that bug a lot. Is the source of the bug known?

teor2345 · 2021-04-01T00:17:13Z

I haven't tested anything more recent, but I'll do that over the next few days.

Recent nightlies are also affected for me.

dhardy · 2021-04-02T21:12:52Z

Worse yet I've found my builds not updating and cargo test running outdated code (using 1.52.0-beta.2).

Aaron1011 · 2021-04-03T02:45:14Z

The underlying issue is tracked at #83538

If you're running into this issue, you can use cargo clean -p <your_package_name> as a temporary workaround.

carado · 2021-04-04T02:26:08Z

(for anyone running into this issue often enough that cargo clean -p is annoying to run every time, if you don't mind the recompilation time you can use export CARGO_INCREMENTAL=0 to disable incremental compilation)

a1ph · 2021-05-06T18:47:31Z

This one reached 1.52.0 stable. Pretty inconvenient.

aurexav · 2021-05-11T04:24:38Z

~~Fixed since nightly-2021-05-10~~

Sry, still found this in 2021-05-11

jyn514 · 2021-05-19T09:46:48Z

This is likely #85360.

Igosuki · 2021-09-12T13:45:22Z

Getting this error too

thread 'rustc' panicked at 'Found unstable fingerprints for evaluate_obligation(c9d69c0d5a400ccc-62c332b1848d388e): Ok(EvaluatedToOkModuloRegions)', /rustc/8c2b6ea37d7719a0370bd404030eef9702c1752c/compiler/rustc_query_system/src/query/plumbing.rs:632:13

earthengine · 2022-09-01T01:34:35Z

I receive this error with

>rustc --version --verbose
rustc 1.63.0 (4b91a6ea7 2022-08-08)
binary: rustc
commit-hash: 4b91a6ea7258a947e59c6522cd5898e7c0a6a88f
commit-date: 2022-08-08
host: x86_64-pc-windows-msvc
release: 1.63.0
LLVM version: 14.0.5

Trying to replicate with simpler code. (But no, the problem is gone when I deleted the target folder and build again)

istankovic · 2023-10-22T14:18:19Z

@AurevoirXavier is this still relevant?

kpreid · 2023-12-25T17:31:06Z

Triage: Relabeling issues which don't have a runnable reproduction (as opposed to having a non-minimized one) to the new label S-needs-repro.
@rustbot label +S-needs-repro -E-needs-mcve

aurexav added C-bug Category: This is a bug. I-ICE Issue: The compiler panicked, giving an Internal Compilation Error (ICE) ❄️ T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Mar 19, 2021

jyn514 added E-needs-mcve Call for participation: This issue has a repro, but needs a Minimal Complete and Verifiable Example A-query-system Area: The rustc query system (https://rustc-dev-guide.rust-lang.org/query.html) labels Mar 19, 2021

jyn514 added A-incr-comp Area: Incremental compilation and removed A-query-system Area: The rustc query system (https://rustc-dev-guide.rust-lang.org/query.html) labels Mar 19, 2021

Aaron1011 changed the title ~~error: internal compiler error: unexpected panic~~ error: found unstable fingerprints for evaluate_obligation Mar 19, 2021

aurexav closed this as completed Mar 25, 2021

aurexav reopened this Mar 25, 2021

dhardy mentioned this issue Apr 1, 2021

Examples not compiling kas-gui/kas#181

Closed

jebrosen mentioned this issue Apr 6, 2021

current nightly doesn't build example rwf2/Rocket#1602

Closed

aurexav mentioned this issue Apr 8, 2021

Use sr-tool for Release darwinia-network/darwinia#632

Merged

morriswinkler mentioned this issue Apr 13, 2021

can not compile rocket example #83741

Closed

lights0123 mentioned this issue May 7, 2021

Compiler panics on 1.52.0 after cargo run and any file edit #85003

Closed

ohsayan mentioned this issue May 7, 2021

unstable fingerprints for item_children - ICE on Rust 1.52.0 stable #85019

Closed

mautamu mentioned this issue May 7, 2021

compiler 1.52.0 error with dependency oracle="0.5" #85028

Closed

gregdhill mentioned this issue May 7, 2021

Internal compiler error - unstable fingerprints interlay/interbtc#103

Closed

wesleywiser mentioned this issue May 10, 2021

Unstable fingerprints tracking issue #84970

Open

aurexav closed this as completed May 11, 2021

aurexav reopened this May 11, 2021

neysofu mentioned this issue Apr 7, 2022

Tracking issue for re-enabling incremental compilation graphprotocol/graph-node#3442

Open

rustbot added S-needs-repro Status: This issue has no reproduction and needs a reproduction to make progress. and removed E-needs-mcve Call for participation: This issue has a repro, but needs a Minimal Complete and Verifiable Example labels Dec 25, 2023

error: found unstable fingerprints for evaluate_obligation #83291

error: found unstable fingerprints for evaluate_obligation #83291

Comments

aurexav commented Mar 19, 2021

Code

Meta

Error output

jyn514 commented Mar 19, 2021

Uh oh!

aurexav commented Mar 19, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Aaron1011 commented Mar 19, 2021

Uh oh!

aurexav commented Mar 25, 2021

Uh oh!

teor2345 commented Mar 27, 2021

Uh oh!

aurexav commented Mar 27, 2021

Uh oh!

real-felix commented Mar 31, 2021

Uh oh!

teor2345 commented Apr 1, 2021

Uh oh!

dhardy commented Apr 2, 2021

Uh oh!

Aaron1011 commented Apr 3, 2021

Uh oh!

carado commented Apr 4, 2021

Uh oh!

a1ph commented May 6, 2021

Uh oh!

aurexav commented May 11, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jyn514 commented May 19, 2021

Uh oh!

Igosuki commented Sep 12, 2021

Uh oh!

earthengine commented Sep 1, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

istankovic commented Oct 22, 2023

Uh oh!

kpreid commented Dec 25, 2023

Uh oh!

aurexav commented Mar 19, 2021 •

edited

Loading

aurexav commented May 11, 2021 •

edited

Loading

earthengine commented Sep 1, 2022 •

edited

Loading