Configuring Spring MVC for AJAX to enable SEO

AJAX based web applications always have challenges with Search engines. The fundamental aspect of AJAX is the content bound to AJAX components are not statically bound when the page loads, but are partially rendered as and when a page events occur and loaded in Javascript DOM tree. Search engines like Google and Bing only indexes static contents.

One of our applications has been built using modern web technologies and was completely Ajax based. We extensively used Backbone.js and other AJAX based technologies. And our application was not optimized for Search engines. As our application was built around Spring MVC so as a first step, I wanted to explore if Spring supports SEO. I found a blog from MKYong Spring 3 MVC ContentNegotiatingViewResolver example. We tried this example and it was slightly complicated to setup our application using this technique.

On further research we found an interesting article in Google Webmasters link. As per this link for every request if we pass a “_escaped_fragment_” the Search Engine are capable of indexing these links. In order enable this in a Spring MVC application; we need to use a framework like Phantom.js. Phantom.js has a capability where in it will take all the data in a DOM tree and output HTML content.

As a 1st step create a Phantom.js script as below,

var page = require('webpage').create();
var system = require('system');

// Open the page
page.open(system.args[1], function (status) {
console.log(page.content);
phantom.exit();
});

We need to execute this code from Java and pass the server returned content to this code as below,

public static String contentBuilderHelper(String redirectUrl){
pbCommand.add("phantom-server.js");
pbCommand.add(redirectUrl);
ProcessBuilder builder = new ProcessBuilder(pbCommand);

final Process process = builder.start();
InputStream is = process.getInputStream();
InputStreamReader isr = new InputStreamReader(is);
BufferedReader br = new BufferedReader(isr);
String line;
while ((line = br.readLine()) != null) {
output = output + line;
}
return output;
}

Configure Spring MVC component is we need to create a custom filter as below,

public class CrawlerFilter implements Filter{
...
public void doFilter(ServletRequest request, ServletResponse response,
FilterChain chain) throws IOException, ServletException {
HttpServletRequest httpRequest = (HttpServletRequest) request;
if(httpRequest.getParameter("_escaped_fragment_")!=null){
String redirectUrl = httpRequest.getRequestURL().toString()+"#!"+httpRequest.getParameter("_escaped_fragment_");
redirectUrl = URLDecoder.decode(redirectUrl,"UTF-8");
System.out.println(redirectUrl);
try {
//Get content from the web
String content = contentBuilderHelper(redirectUrl);
response.setContentType("text/html");
PrintWriter out = response.getWriter();
out.write(content);
out.close();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
chain.doFilter(request, response);
}
...
}

Configuring filter in web.xml

<filter>
<filter-name>GoSmarter Crawler Filter</filter-name>
<filter-class>com.example.filter.CrawlerFilter</filter-class>
</filter>

<filter-mapping>
<filter-name>Crawler Filter</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>

Finally, we also need to configure Phantom.js framework in Tomcat server.

I hope this blog helped you.

About these ads

2 thoughts on “Configuring Spring MVC for AJAX to enable SEO

  1. Nigina

    Thanks for this example, but how i can configure Phantomjs framework in Tomcat server??? I use spring and need to run phantomjs. Please help.

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s