Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

fix_early_postprocessor_bug #7752

Closed

Conversation

owen200008
Copy link
Contributor

@owen200008 owen200008 commented May 13, 2021

What is the purpose of the change

fix bug
early post processor must not Registry beanDefinition
BeanPostProcessorChecker check error, because it think the beanprocess size is less than create. so must have some beanprocess not deal with the bean
see spring error:
same
apache/dubbo-spring-boot-project#786

Brief changelog

Verifying this change

  • Make sure there is a GitHub_issue field for the change (usually before you start working on it). Trivial changes like typos do not require a GitHub issue. Your pull request should address just this issue, without pulling in other changes - one PR resolves one issue.
  • Format the pull request title like [Dubbo-XXX] Fix UnknownException when host config not exist #XXX. Each commit in the pull request should have a meaningful subject line and body.
  • Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
  • Write necessary unit-test to verify your logic correction, more mock a little better when cross module dependency exist. If the new feature or significant change is committed, please remember to add sample in dubbo samples project.
  • Run mvn clean install -DskipTests=false & mvn clean test-compile failsafe:integration-test to make sure unit-test and integration-test pass.
  • If this contribution is large, please follow the Software Donation Guide.

@codecov-commenter
Copy link

codecov-commenter commented May 13, 2021

Codecov Report

Merging #7752 (e7034c0) into master (30616ea) will increase coverage by 0.69%.
The diff coverage is 66.66%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master    #7752      +/-   ##
============================================
+ Coverage     59.23%   59.92%   +0.69%     
- Complexity      530      544      +14     
============================================
  Files          1076     1090      +14     
  Lines         43540    43843     +303     
  Branches       6363     6403      +40     
============================================
+ Hits          25789    26271     +482     
+ Misses        14888    14625     -263     
- Partials       2863     2947      +84     
Impacted Files Coverage Δ
...s/factory/config/DubboEarlyApplicationContext.java 58.33% <58.33%> (ø)
...g/DubboConfigEarlyInitializationPostProcessor.java 44.44% <100.00%> (-2.44%) ⬇️
...pache/dubbo/config/spring/util/DubboBeanUtils.java 54.54% <100.00%> (ø)
...ookeeper/ZookeeperDynamicConfigurationFactory.java 0.00% <0.00%> (-100.00%) ⬇️
...org/apache/dubbo/remoting/zookeeper/EventType.java 0.00% <0.00%> (-61.91%) ⬇️
...pport/zookeeper/ZookeeperDynamicConfiguration.java 0.00% <0.00%> (-60.00%) ⬇️
.../configcenter/support/zookeeper/CacheListener.java 0.00% <0.00%> (-56.25%) ⬇️
...a/org/apache/dubbo/rpc/filter/AccessLogFilter.java 28.35% <0.00%> (-50.75%) ⬇️
...fig/configcenter/TreePathDynamicConfiguration.java 31.42% <0.00%> (-48.58%) ⬇️
...va/org/apache/dubbo/rpc/support/AccessLogData.java 53.16% <0.00%> (-37.98%) ⬇️
... and 164 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 30616ea...e7034c0. Read the comment docs.

@AlbumenJ
Copy link
Member

LGTM. Would you please add some unit test cases for this?

@zhongxiongzeng
Copy link

why not use BeanFactoryAware ?

@owen200008
Copy link
Contributor Author

DubboEarlyApplicationContext implements BeanFactoryAware or DubboConfigEarlyInitializationPostProcessor implements BeanFactoryAware?

@owen200008
Copy link
Contributor Author

DubboConfigEarlyInitializationPostProcessor need early add in BeanPostProcessor.
i think in the step BeanDefinitionRegistry is good way.

@owen200008
Copy link
Contributor Author

it is like other early PostProcessor. see ImportAwareBeanPostProcessor

@kylixs
Copy link
Member

kylixs commented May 27, 2021

@owen200008 This test case is not a normal user scenario, and it seems difficult to understand.
From the description, it seems that AOP is unavailable, please add a test case for specific scenario.

@kylixs
Copy link
Member

kylixs commented May 27, 2021

I don't understand why the registration of DubboConfigEarlyInitializationPostProcessor should be adjusted in this pr. For what?

@owen200008
Copy link
Contributor Author

it means maybe aop is not available。
because beanprocess not total process for the bean。
this is beanprocesscheck meaning of existence

@owen200008
Copy link
Contributor Author

early beanprocess shoud not registry into beanDefinition.
if you understand please see the implement ImportAwareBeanPostProcessor
or other spring early beanprocess implement

@owen200008
Copy link
Contributor Author

is my fault. english is not so good. make puzzle.
BeanPostProcessorChecker check error, because it think the beanprocess size is less than create. so must have some beanprocess not deal with the bean. (so maybe aop beanprocess is not deal with bean, so it have the infomation ( is not eligible for getting processed by all BeanPostProcessors))

@EnableDubboConfig
@PropertySource("classpath:/META-INF/issue-7752-test.properties")
@DirtiesContext(classMode = DirtiesContext.ClassMode.AFTER_EACH_TEST_METHOD)
public class Issue7752Test {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you provide a test for a general application scenario?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, i will try

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will make it clearer what problem is solved, and it will be easier to understand.

@kylixs
Copy link
Member

kylixs commented May 31, 2021

If you just want to register DubboConfigEarlyInitializationPostProcessor earlier, you only need to modify the place where it is registered. It is not recommended to add a wrapper, which makes the registration logic difficult to understand. For example, register it in BeanDefinitionRegistryPostProcessor#postProcessBeanDefinitionRegistry.

@kylixs
Copy link
Member

kylixs commented May 31, 2021

DubboConfigEarlyInitializationPostProcessor is used to solve the problem that AbstractConfig.addIntoConfigManager() post construct does not take effect, because the config bean may be initialized before CommonAnnotationBeanPostProcessor is loaded, and @PostConstruct annotation cannot be processed at this time.

    @PostConstruct
    public void addIntoConfigManager() {
        ApplicationModel.getConfigManager().addConfig(this);
    }

The processing of DubboConfigEarlyInitializationPostProcessor is not elegant enough.
In Dubbo3, this premature initialization problem has been solved with another solution, so it has been deleted.

@chickenlj
Copy link
Contributor

Please follow @kylixs 's instructions to further enhance this patch.

@owen200008
Copy link
Contributor Author

@kylixs @chickenlj
Thank you for telling me the cause and effect.
later i will see dubbo3 how to solve.

DubboConfigEarlyInitializationPostProcessor must not be implements BeanDefinitionRegistryPostProcessor#postProcessBeanDefinitionRegistry.

@owen200008
Copy link
Contributor Author

spring code is not elegant enough.
BeanPostProcessorChecker is the process for check all beanprocessor is deal with beans.
it calc the BeanPostProcessor size code:
String[] postProcessorNames = beanFactory.getBeanNamesForType(BeanPostProcessor.class, true, false);

	// Register BeanPostProcessorChecker that logs an info message when
	// a bean is created during BeanPostProcessor instantiation, i.e. when
	// a bean is not eligible for getting processed by all BeanPostProcessors.
	int beanProcessorTargetCount = beanFactory.getBeanPostProcessorCount() + 1 + postProcessorNames.length; 

the DubboConfigEarlyInitializationPostProcessor in the getBeanPostProcessorCount and in the postProcessorNames.

so the size > real BeanPostProcessor size

@kylixs
Copy link
Member

kylixs commented Jun 10, 2021

I saw this error elsewhere.

 xxxx is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)

It should be that new beans were created and registered during the execution of registerBeanPostProcessors(). These new beans cannot be processed by all BeanPostProcessors because there are still some BeanPostProcessors that are not registered. BeanPostProcessorChecker is used to detect this problem.

So, I think it can be broken down into several things:

  1. Provide a general test case to reproduce this problem
  2. Fix it, that is, find out the beanPostProcessor causing the problem, and avoid creating new beans when it is initialized.

@owen200008
Copy link
Contributor Author

@kylixs you are right.
so i think it is need to create DubboEarlyApplicationContext for add any early BeanPostProcessor.
is it ok i add some test for reproduct this problem and to check?
if you agree, later i will add

@kylixs
Copy link
Member

kylixs commented Jun 15, 2021

@owen200008 I tested this pr locally. First, I removed the code related to DubboEarlyApplicationContext and registered DubboConfigEarlyInitializationPostProcessor in the original way, and then ran Issue7752Test. The result was successful but did not fail, which means that the xxx is not eligible for getting processed by all BeanPostProcessors problem was not triggered.

  1. To find the cause of the problem, we need a test case that can reproduce the problem.
  2. The name of DubboEarlyApplicationContext may be ambiguous, it is a PostProcessor, not an ApplicationContext.

@kylixs
Copy link
Member

kylixs commented Jun 16, 2021

This problem was reproduced in MultipleServicesWithMethodConfigsTest.
The reason for the BeanPostProcessorChecker alarm is that DubboConfigEarlyInitializationPostProcessor is calculated twice, but in fact the same bean registered twice will only keep one, so the BeanPostProcessorCount is one less than expected.

Two registered locations:

  1. Because DubboConfigEarlyInitializationPostProcessor implements BeanDefinitionRegistryPostProcessor, register itself directly in postProcessBeanDefinitionRegistry() or postProcessBeanFactory()
  private void initBeanFactory() {
      if (beanFactory != null) {
          // Register itself
          ...
          beanFactory.addBeanPostProcessor(this);
      }
  }
  1. Because DubboConfigEarlyInitializationPostProcessor also implements the BeanPostProcessor interface and is registered to beanFactory, Spring will register it again as a normal BeanPostProcessor.
	//org.springframework.context.support.PostProcessorRegistrationDelegate
	public static void registerBeanPostProcessors(
			ConfigurableListableBeanFactory beanFactory, AbstractApplicationContext applicationContext) {

		String[] postProcessorNames = beanFactory.getBeanNamesForType(BeanPostProcessor.class, true, false);

		// Register BeanPostProcessorChecker that logs an info message when
		// a bean is created during BeanPostProcessor instantiation, i.e. when
		// a bean is not eligible for getting processed by all BeanPostProcessors.
		int beanProcessorTargetCount = beanFactory.getBeanPostProcessorCount() + 1 + postProcessorNames.length;
		beanFactory.addBeanPostProcessor(new BeanPostProcessorChecker(beanFactory, beanProcessorTargetCount));
		....
	}

The same bean registered twice will only keep one:

        // org.springframework.beans.factory.support.AbstractBeanFactory#addBeanPostProcessor
	public void addBeanPostProcessor(BeanPostProcessor beanPostProcessor) {
		Assert.notNull(beanPostProcessor, "BeanPostProcessor must not be null");
		// Remove from old position, if any
		this.beanPostProcessors.remove(beanPostProcessor);
		// Track whether it is instantiation/destruction aware
		if (beanPostProcessor instanceof InstantiationAwareBeanPostProcessor) {
			this.hasInstantiationAwareBeanPostProcessors = true;
		}
		if (beanPostProcessor instanceof DestructionAwareBeanPostProcessor) {
			this.hasDestructionAwareBeanPostProcessors = true;
		}
		// Add to end of list
		this.beanPostProcessors.add(beanPostProcessor);
	}

Solution:
Ensure that DubboConfigEarlyInitializationPostProcessor can only be registered once.
It is designed to be used before CommonAnnotationBeanPostProcessor is loaded, it needs to be registered to beanFactory.beanPostProcessors as early as possible.

So the solution is keep the first registration, but remove the second registration, that is not to register it as a common bean to beanFactory to avoid repeated registration errors.

The improvement of this PR is generally right, but the name of DubboEarlyApplicationContext needs to be changed, such as DubboEarlyRegistrationPostProcessor.

Also add a use case to reproduce the problem, it is estimated that a common dubbo application example is enough.

@owen200008
Copy link
Contributor Author

#9397 fix it

@owen200008 owen200008 closed this Feb 17, 2022
@owen200008 owen200008 deleted the fix_early_postprocessor_bug branch May 20, 2022 06:11
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants