My SpringBoot app is connecting to a Redis sharded cluster in Azure for better scalability.

Using Jedis to set up the connection and the RedisClusterConfiguration class, my app is unable to perform

Redis any commands: redis.clients.jedis.exceptions.JedisMovedDataException: MOVED

If Jedis is not proper framework for a sharded Redis cluster, what is the alternative framework and where could I find a reference implementation?

My config class:

@Bean

public RedisConnectionFactory connectionFactory() {

return new JedisConnectionFactory(getClusterConfiguration());

}

@Bean

public RedisClusterConfiguration getClusterConfiguration() {

Map<String, Object> source = Maps. newHashMap ();

source.put("spring.redis.cluster.nodes", "xxxx.redis.cache.windows.net:6380");

source.put("spring.redis.cluster.timeout", "30");

source.put("spring.redis.cluster.password", "xxxxxx");

return new RedisClusterConfiguration(new MapPropertySource("RedisClusterConfiguration", source));

}

yeah, but Azure does not give me access to the underlying nodes . Azure Redis has a load balancer in front of the sharded cluster.

This does not work either:

@Bean

public RedisConnectionFactory connectionFactory() {

JedisConnectionFactory conn = new JedisConnectionFactory();

conn.setHostName("ds1-redic-poc.redis.cache.windows.net");

conn.setPassword("GfZ417sFRDlMBR7yqsO0611oaEN5L6SiANjQujoz7WE=");

conn.setPort(6380);

conn.setUseSsl(true);

conn.setUsePool(true);

conn.setTimeout(socketTO);

JedisPoolConfig config = new JedisPoolConfig();

config.setMaxIdle(maxIdle);

config.setMaxTotal(maxTotal);

config.setMinIdle(minIdle);

config.setMaxWaitMillis(maxWaitMillis);

config.setBlockWhenExhausted(blockWhenExhausted);

conn.setPoolConfig(config);

return conn;

}

Does Springboot support lettuce?

>Does Springboot support lettuce?

Yes. It looks like it should be the default, according to this. At least in 2.0 + 2.1.

https://docs.spring.io/spring-boot/docs/2.1.2.RELEASE/reference/htmlsingle/#boot-features-redis

If it's not being picked up check and see if you have Jedis being pulled in somewhere and maybe try excluding the dependency. Also, make sure Lettuce is being pulled in as a dependency and not excluded. If it's on the class path it should use Lettuce.

Daniel - Managing the library dependencies is not as smooth as that. A full reference iplementation/sample project would be very helpful.

org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.data.redis.connection.RedisConnectionFactory]: Factory method 'connectionFactory' threw exception; nested exception is java.lang.NoClassDefFoundError: com/lambdaworks/redis/RedisException

--- gradle build ----

compile group: 'org.springframework.boot', name: 'spring-boot-starter-data-redis'

compile group: 'com.lambdaworks', name: 'lettuce' , version: '2.3.3'

compile group: 'io.lettuce', name: 'lettuce-core', version: '5.0.0.RELEASE'

compile 'redis.clients:jedis'

--- config class --

@Bean

public PdfLocationRepositoryCustom pdfLocationRepositoryImpl(){

return new PdfLocationRedisRepositoryImpl(redisTemplate());

}

@Bean

public RedisConnectionFactory connectionFactory() {

return new LettuceConnectionFactory(getRedisClusterConfiguration());

//return new JedisConnectionFactory(getRedisClusterConfiguration());

}

@Bean

public RedisClusterConfiguration getRedisClusterConfiguration() {

Map<String, Object> source = Maps. newHashMap ();

source.put("spring.redis.cluster.nodes", "ds1-redic-poc.redis.cache.windows.net:6380");

source.put("spring.redis.cluster.timeout", "30");

//source.put("spring.redis.cluster.max", ?);

source.put("spring.redis.cluster.password", "xxxxxx");

//source.put("spring.redis.cluster.max-redirects", ?);

return new RedisClusterConfiguration(new MapPropertySource("RedisClusterConfiguration", source));

}

@Bean

public RedisTemplate<String, Object> redisTemplate() {

RedisTemplate<String, Object> template = new RedisTemplate<>();

template.setConnectionFactory(connectionFactory());

template.setKeySerializer(new StringRedisSerializer());

template.setHashValueSerializer(new GenericToStringSerializer<>( Object.class ));

return template;

}

With all these libs checked in gradle:

compile group: 'org.springframework.boot', name: 'spring-boot-starter-data-redis'

//compile group: 'org.apache.commons', name: 'commons-pool2'

compile group: 'com.lambdaworks', name: 'lettuce' , version: '2.3.3'

compile group: 'biz.paluch.redis', name: 'lettuce', version: '4.0.Final'

compile group: 'io.lettuce', name: 'lettuce-core', version: '5.0.0.RELEASE'

compile 'org.springframework.data:spring-data-redis'

compile 'redis.clients:jedis'

Hmm, it seems like there could be multiple things that cause this. I did find this Github issue which seems to indicate that Lettuce 4.2+ supports Azure Redis Clusters. If you're on 4.5 like in my demo app that should be good enough.

https://github.com/lettuce-io/lettuce-core/issues/246

You might need to hunt for more clues as to what's failing. Maybe try turning up the log level for `org.springframework.data` and/or Lettuce (not sure the package name off hand). Similarly, you might try `tcpdump` or Wireshark to capture the traffic and dig in more, although that's challenging if TLS is involved.

Aside from that Pivotal doesn't support Lettuce itself, so if there is a core issue with it, I would suggest opening a Github issue. The author seems quite active.

Hope that helps!

Finally getting my app with Lettuce running in PCF / Azure.

Facing now a new issue and investigating:

Caused by: org.springframework.data.redis.RedisSystemException: Redis exception; nested exception is com.lambdaworks.redis.RedisException: Cannot retrieve initial cluster partitions from initial URIs [RedisURI [host='ds1-redic-poc.redis.cache.windows.net', port=6380]]

Ok - I got it to connect now. Needed to set SSL to true. Getting a strange memory exception now:

com.lambdaworks.redis.RedisException: io.netty.handler.codec.EncoderException: io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 16776960 byte(s) of direct memory (used: 66816, max: 10485760)

OK, looks like it's using Direct Memory. The Java buildpack defaults the max direct memory to 10M. It looks like you will need more than that. You can bump that up and it should help. See the two sections here.

https://github.com/cloudfoundry/java-buildpack/blob/master/docs/jre-open_jdk_jre.md#java-options

From what I read earlier, it didn't seem like there was a standard in terms of how servers are configured. It could be something MS is doing in their implementation that's just not compatible with the clients. Java is a big space though so you'd think they could provide one Java Redis client that works. Jedis & Lettuce are two popular clients, so if neither works, it sounds like it's time to check with Microsoft Support & see what they have to say.

Post back if you're able to work out a solution or have more questions. Happy to help where we can from the Pivotal side.

Dan

Bad news. I am back to square one . Now that I have Lettuce connecting to an Azure Redis cache with 2 shards, I am getting the same exception I was getting with the other framework, Jedis:

org.springframework.data.redis.RedisSystemException: Error in execution; nested exception is com.lambdaworks.redis.RedisCommandExecutionException: MOVED 13292 52.167.224.109:15003

If I connect my app to an Azure Redis cache with 1 shard, I am able to run different CRUD operations against it. (I also tried to use the non SSL port and that did not help. )

It is very disappointing as I followed Microsoft's recommended solution to use Lettuce.

A tutorial like this one (https://docs.microsoft.com/en-us/java/azure/spring-framework/configure-spring-boot-initializer-java-app-with-redis-cache?view=azure-java-stable) is helpful for a simple initial deployment but is not enough to demonstrate how Azure cache can support an Elastic Cache solution when

Azure does not provide a mechanism to access the underlying Redis nodes that seemed to be needed by these frameworks. (https://github.com/lettuce-io/lettuce-core/blob/5.1.3.RELEASE/src/test/java/io/lettuce/examples/ConnectToMasterSlaveUsingElastiCacheCluster.java)

THoughts?

Unless Microsoft is able to demonstrate with a reference implementation that we can use the Elastic Cache, we are going to have to look at the alternative approach with Redis Lab on our side.

Any others thoughts on your side?

Dan

Do you know if there is a way to configure the LettuceConnection with both a cluster and a pool object?

LettuceConnectionFactory clientConfig = new LettuceConnectionFactory(lettucePool); // DefaultLettucePool

LettuceConnectionFactory factory = new LettuceConnectionFactory(clusterConfig); //RedisClusterConfiguration

Dear All,

I appreciate if you could provide if you could get Jedis or Lettuce working with Azure Redis cache service in Pivotal Cloud Foundry to create pooling configuration. Please advice if Jedis/Lettuce is supported in PCF ?

I am using following configuration:

private JedisPoolConfig buildPoolConfig() {

final JedisPoolConfig poolConfig = new JedisPoolConfig();

poolConfig.setMaxTotal(20);

poolConfig.setMaxIdle(20);

poolConfig.setMinIdle(5);

poolConfig.setTestOnBorrow(true);

poolConfig.setTestOnReturn(true);

poolConfig.setTestWhileIdle(true);

poolConfig.setMinEvictableIdleTimeMillis(Duration.ofSeconds(20).toMillis());

poolConfig.setTimeBetweenEvictionRunsMillis(Duration.ofSeconds(20).toMillis());

poolConfig.setNumTestsPerEvictionRun(-1);

poolConfig.setBlockWhenExhausted(true);

return poolConfig;

Also using in the application.yml:

redis:

jedis:

pool:

max-active: 3

max-idle: 3

max-wait: 1

min-idle: 1

timeout: 3000

Thanks in advance.

Best Regards,

OP

@Oscar P - There is nothing specific to PCF in how you configure a Jedis/Lettuce connection pool. It is the same process on PCF as on your laptop or on a server/VM. The only difference is that you may choose to use different values for development on your local machine vs production on PCF.

https://docs.spring.io/spring-boot/docs/2.2.x/reference/html/spring-boot-features.html#boot-features-redis

See also all the properties you can set to configure things (Find in Page "redis" on the link below):

https://docs.spring.io/spring-boot/docs/2.2.x/reference/html/appendix-application-properties.html#data-properties

In addition, you can choose to get Redis configuration from a bound service. That is typically only for things like host, port and creds. Your pool configuration would not typically come from the bound service.

As an FYI, if you want to pull bound service info, you can use https://github.com/pivotal-cf/java-cfenv. That's the recommended approach.

Hope that helps!

PS. Please start a new thread if you have more questions.