Building a distributed SDL Web Experience Optimization / Fredhopper solution

This post describes a solution for creating a distributed Fredhopper setup for a real-world Production environment. This solution is composed of the following Fredhopper instances:

PRODUCTION "PROD-ORIGIN" INSTANCES

• Fredhopper ProdOrigin Indexer (on "prod-origin-01" server)
• Fredhopper ProdOrigin Query 1 (on "prod-origin-01" server)
• Fredhopper ProdOrigin Query 2 (on "prod-origin-02" server)

PRODUCTION "PROD-REPLICATION" INSTANCES

• Fredhopper ProdReplication Indexer (on "prod-replication-01" server)
• Fredhopper ProdReplication Query 1 (on "prod-replication-01" server)
• Fredhopper ProdReplication Query 2 (on "prod-replication-02" server)

Consider all these instances running on Linux servers with the ProdOrigin and ProdReplication instances available from separate data centers.

There is no "out of the box" mechanism for replicating the published configurations (XML files) between the two Production environments. A bespoke automated script is registered on both Production Indexer instances as follows:

cd /home/fredhopper/fredhopper
./bin/deployment-agent-client set-option indexer /com/fredhopper/config/VersionControl/@customer-publish-action=/fredhopper/indexer/bin/post-publish.sh

This script copies the published business.xml to the remote instance and reloads it by invoking the http://${TARGET}:${HTTP_PORT}/fredhopper/sysadmin/reload-config.jsp?select=business URL.

In order for the script to work, a passwordless authentication must be established between the ProdOrigin and ProdReplication servers.

Parameters such as "target", "user" and "httpport" are set in the post-publish.sh script, which then uses these parameter values to invoke the replicate-smarttarget-promotions.sh script. Both these scripts are available from:

post-publish script
replicate-smarttarget-promotions script

Validation

1. In the SDL Web/Tridion CME, navigate to Targeting

2. In the top left, the source drop-down should default to Staging

3. Validate that both the Staging and Production sources load their lists of promotions correctly

4. On source Staging, create a new Promotion called "TESTPROMO" and save it

5. Click the ... button of TESTPROMO and choose Copy To... > Production

6. Change the source drop-down to Production, and validate that TESTPROMO is now available in Production

7. Log in to the ProdReplication Fredhopper server, and validate that the TESTPROMO has been copied by the replication script:

ssh root@prod-replication-01

grep "TESTPROMO" /fredhopper/data/instances/indexer/config/business.xml

If the promotion was replicated, you'll see an XML statement on the command line.

Conclusion

The standard Fredhopper processes continue working as usual on the Production servers. For example, the syncserver/syncclient processes continue sending the published configuration (XML files) from the indexer to the query instances. The replication itself is taken care of by the bespoke replication script as soon as the origin indexer instance is updated. As expected (and required), the replication is between indexer instances and Fredhopper takes care of the internal processing as usual.