{"id":367,"date":"2013-12-19T22:16:49","date_gmt":"2013-12-20T03:16:49","guid":{"rendered":"http:\/\/blog.fritzhardy.com\/?p=367"},"modified":"2014-02-24T01:18:05","modified_gmt":"2014-02-24T06:18:05","slug":"suny-federation-with-simplesamlphp","status":"publish","type":"post","link":"https:\/\/blog.fritzhardy.com\/?p=367","title":{"rendered":"SUNY Federation with SimpleSAMLphp in HA"},"content":{"rendered":"

SimpleSAMLphp<\/a> is the ideal choice as a SAML 2.0 IdP for SUNY Federation. As it will be the core authentication mechanism for SUNY services, high-availability and load-balancing are an important consideration. Here is one approach to achieving this with HAProxy and Keepalived.<\/p>\n

We intend to have a pair (at least) of Apache servers running identical SimpleSAMLphp installations. These will sit behind a pair of load-balancer machines running HAProxy and Keepalived, the first to balance HTTP traffic to the SAML servers, the second to provide virtual IP failover between the balancers. This forward-facing virtual IP will be stunnel-encryped, and advertised as our SAML Identity Provider (IdP).<\/p>\n

__________vip__________
\n______keepalived______
\nbalance1____balance2
\n|____\\__haproxy__\/____|
\nsaml1___________saml2<\/p>\n

Senior sysadmin Greg Kuchyt<\/a> pioneered our use of these technologies for load-balancing and failover of our 389-DS LDAP services in a manner much like this, an approach that has served us well for many years, and which we have used for other services.<\/p>\n

Our SAML setup is based on the document collection in the SUNY IDM Wiki<\/a>, particularly the excellent Installing SimpleSAMLphp for SUNY Federation<\/a> guide. Suffice it to say that I will not go into any detail on our IdP setup, instead covering only what SimpleSAMLphp changes are necessary to share session data for load-balancing and failover.<\/p>\n

Shared Session Data with Memcache<\/strong><\/p>\n

On the SAML servers, in order to pave the way for balancing load across multiple machines, we need to ensure that SimpleSAMLphp session handling is switched over to memcache:<\/p>\n

\r\nyum install memcached php-pecl-memcache\r\nservice memcached start\r\nchkconfig memcached on\r\n<\/pre>\n
Note that SimpleSAMLphp uses PHP memcache and not the newer PHP memcached, but both use the system memcached daemon.<\/p>\n
Firewalls will need to be opened to allow incoming memcache connections on port 11211.<\/p>\n
In the SimpleSAMLphp configuration, set the store.type, and then adjust the memcache_store.servers array to taste (see docs). The following will cause all session data to be saved to the memcache servers on both boxes:<\/p>\n
config.php:<\/p>\n
\r\n'store.type' => 'memcache',\r\n'memcache_store.servers' => array(\r\n        array(\r\n                array('hostname' => 'saml1.potsdam.edu'),\r\n        ),\r\n        array(\r\n                array('hostname' => 'saml2.potsdam.edu'),\r\n        ),\r\n),\r\n<\/pre>\n
Load-Balancing with HAProxy<\/strong><\/p>\n
On the balance servers we first install HAProxy.  The following configuration uses the least connection algorithm to balance HTTP sessions between the two SAML servers behind them.<\/p>\n
\/etc\/haproxy\/haproxy.cfg:<\/p>\n
\r\ndefaults\r\n    mode                    http\r\n    log                     global\r\n    option                  httplog\r\n    option                  dontlognull\r\n    option http-server-close\r\n    option forwardfor       except 127.0.0.0\/8\r\n    option                  redispatch\r\n    retries                 3\r\n    timeout http-request    10s\r\n    timeout queue           1m\r\n    timeout connect         10s\r\n    timeout client          1m\r\n    timeout server          1m\r\n    timeout http-keep-alive 10s\r\n    timeout check           10s\r\n    maxconn                 3000\r\n\r\nlisten https\r\n    balance leastconn\r\n    bind    127.0.0.1:4430\r\n    option  httpclose\r\n    option  forwardfor\r\n    option  httplog\r\n    option  httpchk         GET \/\r\n    server  saml1 saml1.potsdam.edu:80 check inter 5000 downinter 500\r\n    server  saml2 saml2.potsdam.edu:80 check inter 5000 downinter 500\r\n<\/pre>\n
Note that hostnames are used here for clarity, but it is probably preferable to use IP addresses for server config lines.  Also note that we bind our instance to localhost:4430, since we do not intend for public connections to hit this port directly.  Instead they will hit SSL on port 443 provided by stunnel (later).<\/p>\n
Start haproxy:<\/p>\n
\r\nservice haproxy start\r\n<\/pre>\n
We now have two balancers that will each balance HTTP connections to the SAML servers.<\/p>\n
Failover with Keepalived<\/strong><\/p>\n
On the balance servers we next install Keepalived.  On balance1, the following configuration establishes a virtual IP address to be maintained between the balance servers.<\/p>\n
\/etc\/keepalived\/keepalived.conf:<\/p>\n
\r\nglobal_defs {\r\n        notification_email {\r\n                devnull@potsdam.edu\r\n        }\r\n        notification_email_from devnull@potsdam.edu\r\n        smtp_server 10.137.110.104\r\n        smtp_connect_timeout 30\r\n}\r\n\r\nvrrp_instance VI_1 {\r\n        virtual_router_id 1\r\n        state MASTER\r\n        priority 100\r\n        interface eth2\r\n\r\n        smtp_alert\r\n\r\n        authentication {\r\n                auth_type AH\r\n                auth_pass SomeKindofPasswordHere!\r\n        }\r\n        virtual_ipaddress {\r\n                10.137.100.101\/24 brd 10.137.100.255 dev eth2\r\n        }\r\n}\r\n<\/pre>\n
On balance2, we install the same configuration with two key differences:<\/p>\n
\r\n        state BACKUP\r\n        priority 50\r\n<\/pre>\n
Start keepalived:<\/p>\n
\r\nservice keepalived start\r\n<\/pre>\n
We now have a front-facing virtual IP address 10.137.100.101 providing failover between the balancers.  You can view the status of the IP with: ip addr show.<\/p>\n
SSL with Stunnel<\/strong><\/p>\n
On the balance servers we next install stunnel to encrypt communication on the virtual IP port 443.<\/p>\n
\/etc\/stunnel\/https.conf:<\/p>\n
\r\n#CAfile = \/etc\/pki\/tls\/certs\/entrust-chain.crt\r\ncert = \/etc\/pki\/tls\/certs\/saml.potsdam.edu.crt\r\nkey = \/etc\/pki\/tls\/private\/saml.potsdam.edu.key\r\n[https]\r\n        # public virtual ip address\r\n        accept = 10.137.100.101:443\r\n        connect = 127.0.0.1:4430\r\n\tverify = 0\r\n<\/pre>\n
An example self-signed cert:<\/p>\n
\r\nopenssl genrsa -aes256 -out pass.key 2048\r\nopenssl rsa -in pass.key -out server.key\r\nopenssl req -new -key server.key -x509 -out server.crt -days 999\r\n<\/pre>\n
Note that key and cert will need to be copied into the locations referenced above.<\/p>\n
Start stunnel with the following, probably in an init script:<\/p>\n
\r\nstunnel \/etc\/stunnel\/https.conf\r\n<\/pre>\n
With stunnel running, we now have encrypted communication on our virtual IP.<\/p>\n
Hacking SimpleSAMLphp for Offloaded SSL<\/strong><\/p>\n
SimpleSAMLphp makes use of internal functions to determine port number, HTTP versus HTTPS, and to build links based on those determinations.  Because we offloaded SSL to our balancer, it throws off these functions, with the result that links and such are improperly constructed.<\/p>\n
Discussion at https:\/\/groups.google.com\/forum\/#!topic\/simplesamlphp\/m0aqJoURl7I<\/a> suggests that there may be simpler ways to do this, eventually, but I settled on the changes represented in the following patch to lib\/SimpleSAML\/Utilities.php:<\/p>\n
\r\n--- Utilities.dist.php\t2013-04-08 04:44:05.000000000 -0400\r\n+++ Utilities.php\t2013-12-14 15:35:04.000000000 -0500\r\n@@ -87,6 +87,8 @@\r\n \t *\/\r\n \tpublic static function isHTTPS() {\r\n \r\n+return TRUE;\r\n+\r\n \t\t$url = self::getBaseURL();\r\n \r\n \t\t$end = strpos($url,':\/\/');\r\n@@ -105,6 +107,8 @@\r\n \t *\/\r\n \tprivate static function getServerHTTPS() {\r\n \r\n+return TRUE;\r\n+\r\n \t\tif(!array_key_exists('HTTPS', $_SERVER)) {\r\n \t\t\t\/* Not an https-request. *\/\r\n \t\t\treturn FALSE;\r\n@@ -128,6 +132,8 @@\r\n \t *\/\r\n \tprivate static function getServerPort() {\r\n \r\n+return '';\r\n+\r\n \t\tif (isset($_SERVER[\"SERVER_PORT\"])) {\r\n \t\t\t$portnumber = $_SERVER[\"SERVER_PORT\"];\r\n \t\t} else {\r\n<\/pre>\n
After applying the patch, we should find that all clicks everywhere remain based on https:\/\/saml.potsdam.edu.<\/p>\n","protected":false},"excerpt":{"rendered":"
SimpleSAMLphp is the ideal choice as a SAML 2.0 IdP for SUNY Federation. As it will be the core authentication mechanism for SUNY services, high-availability and load-balancing are an important consideration. Here is one approach to achieving this with HAProxy and Keepalived. We intend to have a pair (at least) of Apache servers running identical […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4],"tags":[],"class_list":["post-367","post","type-post","status-publish","format-standard","hentry","category-technology"],"_links":{"self":[{"href":"https:\/\/blog.fritzhardy.com\/index.php?rest_route=\/wp\/v2\/posts\/367","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.fritzhardy.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.fritzhardy.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.fritzhardy.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.fritzhardy.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=367"}],"version-history":[{"count":42,"href":"https:\/\/blog.fritzhardy.com\/index.php?rest_route=\/wp\/v2\/posts\/367\/revisions"}],"predecessor-version":[{"id":544,"href":"https:\/\/blog.fritzhardy.com\/index.php?rest_route=\/wp\/v2\/posts\/367\/revisions\/544"}],"wp:attachment":[{"href":"https:\/\/blog.fritzhardy.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=367"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.fritzhardy.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=367"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.fritzhardy.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=367"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}