Load Balancing HTTP and Web Services

Membrane Monitor and Router can work as loadbalancers and distribute calls to several endpoints. The loadbalancing will provide a performance gain if a lot of clients are accessing a service concurrently. Besides that the router also offers failover for web services.

Figure1: Load Balancing for Web Services

The Architecture of the Loadbalancer

A loadbalancer consists of two components: the LoadBalancingInterceptor and a DispatchingStrategy. The LoadBalancingInterceptor asks the DispatchingStrategy to what endpoint the request message should be sent.

Figure2: Components of the Loadbalancer

Membrane comes with a round robin and a by thread strategy that can be used together with the LoadBalancingInterceptor. You can also plug in your own implementation of the DispatchingStrategy interface that provides a different balancing behavior.

Configuration

You can configure the loadbalancer in the file:

<<Membrane installation dir>>/configuration/monitor-beans.xml

The following listening shows a bean definition for the LoadBalancingInterceptor. The endpoint property contains a list of the endpoint URLs the load should be balanced to. The DispatchingStrategy property is configured with an anonymous instance of the RoundRobinStrategy.

<bean id="roundRobinBalancer" class="com.predic8.membrane.core.interceptor.balancer.LoadBalancingInterceptor">
  <property name="displayName" value="roundRobinBalancer" />
  <property name="endpoints">
    <list>
      <value>www.thomas-bayer.com:80 </value>
      <value>www.thomas-bayer.com </value>
    </list>
  </property>
  <property name="dispatchingStrategy">
    <bean class="com.predic8.membrane.core.interceptor.balancer.RoundRobinStrategy" />
  </property>
</bean>
        

RoundRobinStrategy

This DispatchingStrategy distributes the calls evenly between the endpoints. See figure 1 how ten calls are distributed between three endpoints.

ByThreadStrategy

Some endpoints can process only a limited number of calls concurrently. A service that wraps a library that is not thread save for instance can only serve one client at a time. Then you should limit the number of threads to one. The ByThreadStrategy can limit the number of concurrent calls per service to the value specified for the maxNumberOfThreadsPerEndpointProperty.

<bean id="byThreadBalancer" class="com.predic8.membrane.core.interceptor.balancer.LoadBalancingInterceptor">
  <property name="displayName" value="byThreadBalancer" />
  <property name="endpoints">
    <list>
      <value>www.thomas-bayer.com:80 </value>
      <value>www.thomas-bayer.com </value>
    </list>
  </property>
  <property name="dispatchingStrategy">
    <bean class="com.predic8.membrane.core.interceptor.balancer.ByThreadStrategy">
      <property name="maxNumberOfThreadsPerEndpoint" value="10"/>
      <property name="retryTimeOnBusy" value="1000"/>
    </bean>
  </property>
</bean>