AJAX Cross-Site Scripting with Apache RewriteRule
.AJAX has a significant limitation: for security reasons, it can’t fetch data from sites other than the one the user is on. This makes a number of cool things much harder to do.
One way around this is the script tag hack. But that requires a third party that outputs JavaScript files, whereas what I want is to be able to process XML and/or HTML from that third party.
Another way works if you have access to your own server, the one hosting the page the user will be on when their page wants to get data from the third party site: use an Apache rewrite rule.
That sounded quite simple and elegant to me, so I started looking into it. My use case was the magictraders.com pricelists, and I wanted to turn http://magictraders.com/cgi-bin/query.cgi?list=magic&target=yosei&field=0&operator=re into http://sfmagic.org/mtcardprice/yosei. That would make whatever that request returns available to the user’s browser for processing by JavaScript.
The first step was to turn on proxy functionality in Apache. In some cases this requires uncommenting mod_proxy somewhere in httpd.conf, in others it requires recompiling Apache, and in my (OpenBSD) case it requires uncommenting
LoadModule proxy_module /usr/lib/apache/modules/libproxy.so
(In my version of Apache, it also requires not having a ServerName set in the root of the conf file.)
After doing that, I put this line into the .htaccess file at the domain’s root level:
RewriteRule ^mtcardprice/(.*)$ http://magictraders.com/cgi-bin/query.cgi?list=magic&field=0&operator=re&target=$1 [P]
However, that wasn’t all… as a security measure, my firewall runs restrictive pf.conf rules that block packets outgoing from my web server—in OpenBSD the web server is cordned off from the rest of the machine, and this is intended to minimize damage from have some web service compromised. So I needed to edit pf.conf as well.
I found the first part fairly easily: just allow outbound traffic to magictraders.com:
pass out log proto tcp to magictraders.com user www keep state flags S/SA
I thought that would do it. It didn’t; Apache would give me “host not found” errors. This continued even when I switched pf.conf to refer to the magictraders.com IP address. I banged my head against it for quite some time, noting that if I substituted any for the IP or domain, it all worked. Eventually my brother pointed out to me that while the pf rule was using the IP address, Apache itself was using the domain and so needed access to DNS… which was blocked.
So I added:
pass out log proto { tcp,udp } to [my DNS server] user www keep state flags S/SA
And then it all worked.
Naturally, I’m now thinking of processing the magictraders.com prices in a different way, requiring a daily server pull, but I might still end up using this approach, which I’m sure is going to be useful for certain things in the future, including any JavaScript that pulls from multiple different servers.