Collectives™ on Stack Overflow

Find centralized, trusted content and collaborate around the technologies you use most.

Learn more about Collectives

Teams

Q&A for work

Connect and share knowledge within a single location that is structured and easy to search.

Learn more about Teams

Spring Boot: Attempting to override application.properties using EnvironmentPostProcessor

Ask Question

So I'm trying to use key/values stored in Cosul to override values in application.properties. I tried two things.

1) Using Spring Cloud Consul Config. https://cloud.spring.io/spring-cloud-consul/reference/html/#spring-cloud-consul-config

This worked if I did not have the same key defined in my application.properties. If it was defined in application.properties, the value in the properties file was used in all @Value annotation resolution. This is the opposite of what I wanted.

2) Since the above didn't work, I went on to create a custom EnvironmentPostProcessor. I first tried to build a MapPropertySource and used environment.getPropertySources().addAfter(..). This was the same result as above. I then tried to iterate through all property sources, found the one with a name containing "applicationConfig: [classpath:/application" and either set the property value if it exists or putting a new property value. In addition, I added the MapPropertySource to the same EnumerableCompositePropertySource that the "applicationConfig: [classpath:/application" property source is in.

With either approach, it's always the same result. If the key exists in application.properties, that value is used.

What gives? I'm literally overriding the value in the property sources and I can see the values in the debugger before the PostProcessor finishes doing its thing. How is the application.properties value still getting to the @Value annotations?

Here is my current PostProcessor.

@Order(Ordered.LOWEST_PRECEDENCE)
public class ConsulPropertyPostProcessor implements EnvironmentPostProcessor {
    private static final String PROPERTY_SOURCE_NAME = "applicationConfigurationProperties";
    @Override
    public void postProcessEnvironment(ConfigurableEnvironment environment, SpringApplication application) {
        PropertySource<?> system = environment.getPropertySources().get(SYSTEM_ENVIRONMENT_PROPERTY_SOURCE_NAME);
        ConsulKVService consulKVService = new ConsulKVServiceImpl().instantiateConsulKVServiceImpl((String)system.getProperty("CONSUL_HOST"), (String)system.getProperty("CONSUL_TOKEN"));
        Map<String, Object> map = consulKVService.getConsulKeysAndValuesByPrefix((String)system.getProperty("CONSUL_PREFIX"));
        addOrReplace(environment.getPropertySources(), map);
    private void addOrReplace(MutablePropertySources propertySources, Map<String, Object> map) {
        MapPropertySource target = new MapPropertySource("applicationConfig: [consulKVs]", map);
        if (propertySources.contains(PROPERTY_SOURCE_NAME)) {
            PropertySource<?> applicationConfigurationPropertySources = propertySources.get(PROPERTY_SOURCE_NAME);
            for(EnumerableCompositePropertySource applicationPropertySource : (ArrayList<EnumerableCompositePropertySource>)applicationConfigurationPropertySources.getSource()){
                if(applicationPropertySource.getName() != null
                        && applicationPropertySource.getName().contains("applicationConfig: [profile=")) {
                    for(PropertySource singleApplicationPropertySource : applicationPropertySource.getSource()){
                        if(singleApplicationPropertySource.getName().contains("applicationConfig: [classpath:/application")){
                            for (String key : map.keySet()) {
                                if(map.get(key) != null) {
                                    if (singleApplicationPropertySource.containsProperty(key)) {
                                        ((Properties) singleApplicationPropertySource.getSource())
                                                .setProperty(key, (String) map.get(key));
                                    } else {
                                        ((Properties) singleApplicationPropertySource.getSource()).put(key, (String) map.get(key));
                            break;
                    applicationPropertySource.add(target);
                    break;

Thanks in advance everyone.

EDIT: Tried overriding the onApplicationEvent method of the ApplicationListener class with the same result as above. Here is that code.

@Log4j
public class ConsulProperties implements ApplicationListener<ApplicationEnvironmentPreparedEvent> {
    static ConfigurableEnvironment configurableEnvironment;
    private static final String PROPERTY_SOURCE_NAME = "applicationConfigurationProperties";
    public static ConfigurableEnvironment getConfigurableEnvironment() {
        return configurableEnvironment;
    @Override
    public void onApplicationEvent(ApplicationEnvironmentPreparedEvent event) {
        log.info("Received ApplicationEnvironmentPreparedEvent...");
        ConfigurableEnvironment environment = event.getEnvironment();
        configurableEnvironment = environment;
        Properties props = new Properties();
        ConsulKVService consulKVService = new ConsulKVServiceImpl()
                .instantiateConsulKVServiceImpl((String) configurableEnvironment.getProperty("CONSUL_HOST"),
                        (String) configurableEnvironment.getProperty("CONSUL_TOKEN"));
        Map<String, Object> map = consulKVService.getConsulKeysAndValuesByPrefix((String) configurableEnvironment.getProperty("CONSUL_PREFIX"));
        while(map.values().remove(null));
        addOrReplace(environment.getPropertySources(), map);
    private void addOrReplace(MutablePropertySources propertySources, Map<String, Object> map) {
        MapPropertySource target = new MapPropertySource("applicationConfig: [consulKVs]", map);
        if (propertySources.contains(PROPERTY_SOURCE_NAME)) {
            PropertySource<?> applicationConfigurationPropertySources = propertySources.get(PROPERTY_SOURCE_NAME);
            for(EnumerableCompositePropertySource applicationPropertySource : (ArrayList<EnumerableCompositePropertySource>)applicationConfigurationPropertySources.getSource()){
                if(applicationPropertySource.getName() != null
                        && applicationPropertySource.getName().contains("applicationConfig: [profile=")) {
                    for(PropertySource singleApplicationPropertySource : applicationPropertySource.getSource()){
                        if(singleApplicationPropertySource.getName().contains("applicationConfig: [classpath:/application")){
                            for (String key : map.keySet()) {
                                if (singleApplicationPropertySource.containsProperty(key)) {
                                    ((Properties) singleApplicationPropertySource.getSource())
                                            .setProperty(key, (String) map.get(key));
                                } else {
                                    ((Properties) singleApplicationPropertySource.getSource()).put(key,
                                            map.get(key));
                            applicationPropertySource.add(target);
                            Properties properties = new Properties();
                            properties.putAll(map);
                            propertySources.addLast(new PropertiesPropertySource("consulKVs", properties));
                            break;
                    break;
                Properties defined in application.properties override the properties from other sources. I think you can use event listeners to override the properties. Check thetechnojournals.com/2019/10/… for the event listeners implementations.
– Ashok Prajapati
                Nov 7, 2019 at 14:47
                @AshokPrajapati I tried implementing ApplicationListener<ApplicationEnvironmentPreparedEvent> and using Override on "public void onApplicationEvent(ApplicationEnvironmentPreparedEvent event) " . Did this with similar code to my OP.  Same result though.  Will add this attempt to the OP.
– Jason Emeric
                Nov 7, 2019 at 18:34
                are you sure that your listener is picked and executed by Spring boot? How are you registering it?
– Ashok Prajapati
                Nov 8, 2019 at 12:42
                @AshokPrajapati For sure.  I registered it in spring.factories using "org.springframework.context.ApplicationListener=com.[package name removed for privacy].ConsulProperties", I can see my "Received ApplicationEnvironmentPreparedEvent..." log statement and the breakpoint is triggered on app start.
– Jason Emeric
                Nov 8, 2019 at 15:28

There is a problem with your code where you are adding the new property source using below code. Please note that when you call "addLast" the property source added through this method takes lowest precedence and will never update the already available properties.

propertySources.addLast(new PropertiesPropertySource("consulKVs", properties));

Instead of above you can use "addFirst" to add the property source which should take highest precedence as given in below code. There are some other methods like "addAfter" & "addBefore" also available which you can explore to add the property source at exact location. In any case "addFirst" will take precedence on all other ways, so I think you can use "addFirst" to update the property source.

propertySources.addFirst(new PropertiesPropertySource("consulKVs", properties));

I have tested this scenario using ApplicationEnvironmentPreparedEvent and it working fine. Hope it should resolve your issue.

I flipped to addFirst and I still see the application.properties values in a controller I spun up for a quick verification. – Jason Emeric Nov 8, 2019 at 17:55

It looks like you're trying to change the convention that spring is not designed for. The code you provide is not that easy to maintain and it requires a deep knowledge of Spring internals. Frankly I can't tell without debugging how to achieve what you want, however I have an another approach in mind:

You can use spring profiles in the following way:

Say you have a property db.name=abc in application.properties and db.name=xyz in consul, and I assume your goal is to have db.name=xyz resolved by spring.

In this case, move db.name=abc into application-local.properties and start the application with --spring.profiles.active=local if you want to have the property from the local file, and without this profile if you want to go with consul.

You can even add an active profile dynamically in the EnvironmentPostProcessor (you've got there anyway already), but this is one line of code in EnvironmentPostProcessor.

Agree with your comments. Your "following way" was exactly my first attempt. My cohort who put this story on my board didn't want that though. They are used to overriding application.properties values with env vars set in AWS BeanStalk. We now hit the 4096 byte limit of BeanStalk and want to move off and migrate the kev/values to Consul with the same use case. Drop key/vlaue in Consul to override application.properties. – Jason Emeric Nov 7, 2019 at 18:58

Thanks for contributing an answer to Stack Overflow!

  • Please be sure to answer the question. Provide details and share your research!

But avoid

  • Asking for help, clarification, or responding to other answers.
  • Making statements based on opinion; back them up with references or personal experience.

To learn more, see our tips on writing great answers.