Skip to content

Large TestResults files #33787

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Closed
AndriySvyryd opened this issue May 22, 2024 · 2 comments · Fixed by #33819
Closed

Large TestResults files #33787

AndriySvyryd opened this issue May 22, 2024 · 2 comments · Fixed by #33819

Comments

@AndriySvyryd
Copy link
Member

Files like EFCore.SqlServer.FunctionalTests_net9.0_x64.xml are over 100MB in size because they contain all test output
Possibly caused by #31017

Reducing the amount of output will improve test running time.

@roji
Copy link
Member

roji commented May 22, 2024

Oh weird...! I'll take a look...

@roji
Copy link
Member

roji commented May 27, 2024

Looked into this... The query tests always logged a little bit, but since I merged #31017, at some point we started dumping the entire shaper representation, causing the logs to be much heavier (see below).

In any case, submitting #33819 to not log at all from AssertQuery and friends, unless an error occurs.


Right after #31017 (commit ef30eb2), a basic query test emitted a bit of logging:

Logging at ef30eb2
Compiling query expression: 
'DbSet<Customer>()
    .Where(c => c.City == "London")'

Generated query execution expression: 
'queryContext => new SingleQueryingEnumerable<Customer>(
    (RelationalQueryContext)queryContext, 
    RelationalCommandCache.QueryExpression(
        Projection Mapping:
            EmptyProjectionMember -> Dictionary<IProperty, int> { [Property: Customer.CustomerID (string) Required PK AfterSave:Throw MaxLength(5), 0], [Property: Customer.Address (string) MaxLength(60), 1], [Property: Customer.City (string) Index MaxLength(15), 2], [Property: Customer.CompanyName (string) Required Index MaxLength(40), 3], [Property: Customer.ContactName (string) MaxLength(30), 4], [Property: Customer.ContactTitle (string) MaxLength(30), 5], [Property: Customer.Country (string) MaxLength(15), 6], [Property: Customer.Fax (string) MaxLength(24), 7], [Property: Customer.Phone (string) MaxLength(24), 8], [Property: Customer.PostalCode (string) Index MaxLength(10), 9], [Property: Customer.Region (string) Index MaxLength(15), 10] }
        SELECT c.CustomerID, c.Address, c.City, c.CompanyName, c.ContactName, c.ContactTitle, c.Country, c.Fax, c.Phone, c.PostalCode, c.Region
        FROM Customers AS c
        WHERE c.City == N'London'), 
    ReaderColumn[] { ReaderColumn<object>, ReaderColumn<object>, ReaderColumn<object>, ReaderColumn<object>, ReaderColumn<object>, ReaderColumn<object>, ReaderColumn<object>, ReaderColumn<object>, ReaderColumn<object>, ReaderColumn<object>, ReaderColumn<object> }, 
    Func<QueryContext, DbDataReader, ResultContext, SingleQueryResultCoordinator, Customer>, 
    TestModels.Northwind.NorthwindSqlServerContext, 
    False, 
    True, 
    True
)'

Creating DbCommand for 'ExecuteReader'.

Created DbCommand for 'ExecuteReader' (0ms).

Initialized DbCommand for 'ExecuteReader' (0ms).

Closing data reader to 'Northwind' on server 'localhost'.

A data reader for 'Northwind' on server 'localhost' is being disposed after spending 1ms reading results.

In latest main, the entire shaper is dumped, making for lots of logging:

Logging at 5eec48b
Compiling query expression: 
'DbSet<Customer>()
    .Where(c => c.City == "London")'

Generated query execution expression: 
'queryContext => SingleQueryingEnumerable.Create<Customer>(
    relationalQueryContext: (RelationalQueryContext)queryContext, 
    relationalCommandResolver: parameters => [LIFTABLE Constant: RelationalCommandCache.QueryExpression(
        Projection Mapping:
            EmptyProjectionMember -> Dictionary<IProperty, int> { [Property: Customer.CustomerID (string) Required PK AfterSave:Throw MaxLength(5), 0], [Property: Customer.Address (string) MaxLength(60), 1], [Property: Customer.City (string) Index MaxLength(15), 2], [Property: Customer.CompanyName (string) Required Index MaxLength(40), 3], [Property: Customer.ContactName (string) MaxLength(30), 4], [Property: Customer.ContactTitle (string) MaxLength(30), 5], [Property: Customer.Country (string) MaxLength(15), 6], [Property: Customer.Fax (string) MaxLength(24), 7], [Property: Customer.Phone (string) MaxLength(24), 8], [Property: Customer.PostalCode (string) Index MaxLength(10), 9], [Property: Customer.Region (string) Index MaxLength(15), 10] }
        SELECT c.CustomerID, c.Address, c.City, c.CompanyName, c.ContactName, c.ContactTitle, c.Country, c.Fax, c.Phone, c.PostalCode, c.Region
        FROM Customers AS c
        WHERE c.City == N'London') | Resolver: c => new RelationalCommandCache(
        c.Dependencies.MemoryCache, 
        c.RelationalDependencies.QuerySqlGeneratorFactory, 
        c.RelationalDependencies.RelationalParameterBasedSqlProcessorFactory, 
        <>c__DisplayClass26_0.queryExpression, 
        RelationalShapedQueryCompilingExpressionVisitor._useRelationalNulls
    )].GetRelationalCommandTemplate(parameters), 
    readerColumns: [LIFTABLE Constant: ReaderColumn[] { ReaderColumn<object>, ReaderColumn<object>, ReaderColumn<object>, ReaderColumn<object>, ReaderColumn<object>, ReaderColumn<object>, ReaderColumn<object>, ReaderColumn<object>, ReaderColumn<object>, ReaderColumn<object>, ReaderColumn<object> } | Resolver: namelessParameter{0} => new ReaderColumn[]
    { 
        new ReaderColumn<object>(
            False, 
            null, 
            namelessParameter{0}.Dependencies.Model.FindEntityType("TestModels.Northwind.Customer").FindProperty("CustomerID"), 
            (dataReader, indexMap) => (object)dataReader.GetString(0)
        ), 
        new ReaderColumn<object>(
            True, 
            null, 
            namelessParameter{0}.Dependencies.Model.FindEntityType("TestModels.Northwind.Customer").FindProperty("Address"), 
            (dataReader, indexMap) => (object)dataReader.GetString(1)
        ), 
        new ReaderColumn<object>(
            True, 
            null, 
            namelessParameter{0}.Dependencies.Model.FindEntityType("TestModels.Northwind.Customer").FindProperty("City"), 
            (dataReader, indexMap) => (object)dataReader.GetString(2)
        ), 
        new ReaderColumn<object>(
            False, 
            null, 
            namelessParameter{0}.Dependencies.Model.FindEntityType("TestModels.Northwind.Customer").FindProperty("CompanyName"), 
            (dataReader, indexMap) => (object)dataReader.GetString(3)
        ), 
        new ReaderColumn<object>(
            True, 
            null, 
            namelessParameter{0}.Dependencies.Model.FindEntityType("TestModels.Northwind.Customer").FindProperty("ContactName"), 
            (dataReader, indexMap) => (object)dataReader.GetString(4)
        ), 
        new ReaderColumn<object>(
            True, 
            null, 
            namelessParameter{0}.Dependencies.Model.FindEntityType("TestModels.Northwind.Customer").FindProperty("ContactTitle"), 
            (dataReader, indexMap) => (object)dataReader.GetString(5)
        ), 
        new ReaderColumn<object>(
            True, 
            null, 
            namelessParameter{0}.Dependencies.Model.FindEntityType("TestModels.Northwind.Customer").FindProperty("Country"), 
            (dataReader, indexMap) => (object)dataReader.GetString(6)
        ), 
        new ReaderColumn<object>(
            True, 
            null, 
            namelessParameter{0}.Dependencies.Model.FindEntityType("TestModels.Northwind.Customer").FindProperty("Fax"), 
            (dataReader, indexMap) => (object)dataReader.GetString(7)
        ), 
        new ReaderColumn<object>(
            True, 
            null, 
            namelessParameter{0}.Dependencies.Model.FindEntityType("TestModels.Northwind.Customer").FindProperty("Phone"), 
            (dataReader, indexMap) => (object)dataReader.GetString(8)
        ), 
        new ReaderColumn<object>(
            True, 
            null, 
            namelessParameter{0}.Dependencies.Model.FindEntityType("TestModels.Northwind.Customer").FindProperty("PostalCode"), 
            (dataReader, indexMap) => (object)dataReader.GetString(9)
        ), 
        new ReaderColumn<object>(
            True, 
            null, 
            namelessParameter{0}.Dependencies.Model.FindEntityType("TestModels.Northwind.Customer").FindProperty("Region"), 
            (dataReader, indexMap) => (object)dataReader.GetString(10)
        ) 
    }], 
    shaper: (queryContext, dataReader, resultContext, resultCoordinator) => 
    {
        Customer entity;
        entity = 
        {
            MaterializationContext materializationContext1;
            IEntityType entityType1;
            Customer instance1;
            InternalEntityEntry entry1;
            bool hasNullKey1;
            materializationContext1 = new MaterializationContext(
                [LIFTABLE Constant: ValueBuffer | Resolver: _ => (object)ValueBuffer.Empty], 
                queryContext.Context
            );
            instance1 = default(Customer);
            entry1 = queryContext.TryGetEntry(
                key: [LIFTABLE Constant: Key: Customer.CustomerID PK | Resolver: c => c.Dependencies.Model.FindEntityType("TestModels.Northwind.Customer").FindPrimaryKey()], 
                keyValues: new object[]{ (object)(string)dataReader.GetFieldValue<object>(0) }, 
                throwOnNullKey: True, 
                hasNullKey: hasNullKey1);
            !(hasNullKey1) ? entry1 != default(InternalEntityEntry) ? 
            {
                entityType1 = entry1.EntityType;
                return instance1 = (Customer)entry1.Entity;
            } : 
            {
                ISnapshot shadowSnapshot1;
                shadowSnapshot1 = [LIFTABLE Constant: Snapshot | Resolver: _ => Snapshot.Empty];
                entityType1 = [LIFTABLE Constant: EntityType: Customer | Resolver: namelessParameter{1} => namelessParameter{1}.Dependencies.Model.FindEntityType("TestModels.Northwind.Customer")];
                instance1 = switch (entityType1)
                {
                    case [LIFTABLE Constant: EntityType: Customer | Resolver: namelessParameter{2} => namelessParameter{2}.Dependencies.Model.FindEntityType("TestModels.Northwind.Customer")]: 
                        {
                            return 
                            {
                                Customer instance;
                                DbContext namelessParameter{3};
                                ILazyLoader namelessParameter{4};
                                NorthwindContext namelessParameter{5};
                                namelessParameter{3} = materializationContext1.Context;
                                namelessParameter{4} = InfrastructureExtensions.GetService<ILazyLoader>((IInfrastructure<IServiceProvider>)materializationContext1.Context);
                                namelessParameter{5} = (materializationContext1.Context as NorthwindContext);
                                instance = new Customer(
                                    namelessParameter{3}, 
                                    namelessParameter{4}, 
                                    (string)dataReader.GetFieldValue<object>(0)
                                );
                                instance.<Address>k__BackingField = dataReader.IsDBNull(1) ? default(string) : (string)dataReader.GetFieldValue<object>(1);
                                instance.<City>k__BackingField = dataReader.IsDBNull(2) ? default(string) : (string)dataReader.GetFieldValue<object>(2);
                                instance.<CompanyName>k__BackingField = (string)dataReader.GetFieldValue<object>(3);
                                instance.<ContactName>k__BackingField = dataReader.IsDBNull(4) ? default(string) : (string)dataReader.GetFieldValue<object>(4);
                                instance.<ContactTitle>k__BackingField = dataReader.IsDBNull(5) ? default(string) : (string)dataReader.GetFieldValue<object>(5);
                                instance.<Country>k__BackingField = dataReader.IsDBNull(6) ? default(string) : (string)dataReader.GetFieldValue<object>(6);
                                instance.<Fax>k__BackingField = dataReader.IsDBNull(7) ? default(string) : (string)dataReader.GetFieldValue<object>(7);
                                instance.<Phone>k__BackingField = dataReader.IsDBNull(8) ? default(string) : (string)dataReader.GetFieldValue<object>(8);
                                instance.<PostalCode>k__BackingField = dataReader.IsDBNull(9) ? default(string) : (string)dataReader.GetFieldValue<object>(9);
                                instance.<Region>k__BackingField = dataReader.IsDBNull(10) ? default(string) : (string)dataReader.GetFieldValue<object>(10);
                                instance.Context = namelessParameter{5};
                                (instance is IInjectableService) ? ((IInjectableService)instance).Injected(
                                    context: materializationContext1.Context, 
                                    entity: instance, 
                                    queryTrackingBehavior: TrackAll, 
                                    structuralType: [LIFTABLE Constant: EntityType: Customer | Resolver: namelessParameter{6} => namelessParameter{6}.Dependencies.Model.FindEntityType("TestModels.Northwind.Customer")]) : default(void);
                                (namelessParameter{3} is IInjectableService) ? ((IInjectableService)namelessParameter{3}).Injected(
                                    context: materializationContext1.Context, 
                                    entity: instance, 
                                    queryTrackingBehavior: TrackAll, 
                                    structuralType: [LIFTABLE Constant: EntityType: Customer | Resolver: namelessParameter{7} => namelessParameter{7}.Dependencies.Model.FindEntityType("TestModels.Northwind.Customer")]) : default(void);
                                (namelessParameter{4} is IInjectableService) ? ((IInjectableService)namelessParameter{4}).Injected(
                                    context: materializationContext1.Context, 
                                    entity: instance, 
                                    queryTrackingBehavior: TrackAll, 
                                    structuralType: [LIFTABLE Constant: EntityType: Customer | Resolver: namelessParameter{8} => namelessParameter{8}.Dependencies.Model.FindEntityType("TestModels.Northwind.Customer")]) : default(void);
                                (namelessParameter{5} is IInjectableService) ? ((IInjectableService)namelessParameter{5}).Injected(
                                    context: materializationContext1.Context, 
                                    entity: instance, 
                                    queryTrackingBehavior: TrackAll, 
                                    structuralType: [LIFTABLE Constant: EntityType: Customer | Resolver: namelessParameter{9} => namelessParameter{9}.Dependencies.Model.FindEntityType("TestModels.Northwind.Customer")]) : default(void);
                                return instance;
                            }}
                    default: 
                        default(Customer)
                }
                ;
                entry1 = entityType1 == default(IEntityType) ? default(InternalEntityEntry) : queryContext.StartTracking(
                    entityType: entityType1, 
                    entity: instance1, 
                    snapshot: shadowSnapshot1);
                return instance1;
            } : default(void);
            return instance1;
        };
        return entity;
    }, 
    contextType: TestModels.Northwind.NorthwindSqlServerContext, 
    standAloneStateManager: False, 
    detailedErrorsEnabled: True, 
    threadSafetyChecksEnabled: True)'

Creating DbCommand for 'ExecuteReader'.

Created DbCommand for 'ExecuteReader' (1ms).

Initialized DbCommand for 'ExecuteReader' (3ms).

Closing data reader to 'Northwind' on server 'localhost'.

A data reader for 'Northwind' on server 'localhost' is being disposed after spending 20ms reading results.

@ajcvickers ajcvickers added this to the 9.0.0-preview6 milestone Jun 5, 2024
@roji roji modified the milestones: 9.0.0-preview6, 9.0.0 Oct 12, 2024
# for free to join this conversation on GitHub. Already have an account? # to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants