sebnilsson.com | Liquid Development Is What I Do
Seb Nilsson

ASP.NET MVC SEO-Framework

For any serious web-application you should always implement a solid Search engine optimization-strategy, but there is no standardized way to handle this in ASP.NET MVC, out of the box.

You could easily use the ViewBag-object to send dynamic values from Controller-Actions into Views, like this, for example:

public ActionResult Index()
{
    ViewBag.Title = "This is the page title";

    return View();
}

Then you'd have to make sure you correctly spell or copy-paste ViewBag.Title correctly into your View:

<head>
    <title>@ViewBag.Title</title>
    <!-- More head-values -->
</head>

One problem is that if you refactor the naming for ViewBag.Title into, for example ViewBag.Title, this will break the functionality, potentially site-wide, because you won't get any tooling-help from Visual Studio for the renaming.

This is why I created a framework for ASP.NET MVC SEO, to get structured and reusable functionality around the SEO-data for a web-application. The framework is available on Nuget, with the source-code on GitHub.

Using a SeoHelper-object, available inside Controllers and Views, you can set SEO-related data like:

  • Meta-Description
  • Meta-Keywords
  • Title, split on page-title and base-title (website-title)
  • Canonical Link
  • Meta No-index for robots

This can be done inside Controllers and Controller-actions:

[SeoBaseTitle("Website name")]
public class InfoController : SeoController
{
    [SeoTitle("Listing items")]
    [SeoMetaDescription("List of the company's product-items")]
    public ActionResult List()
    {
        var list = GetList();

        if (list.Any())
        {
            Seo.Title += $" (Total: {list.Count})";
            Seo.LinkCanonical = "~/pages/list.html";
        }
        else
        {
            Seo.MetaRobotsNoIndex = true;
        }

        return View(model);
    }
}

Or inside Views:

@{
    Layout = null;
    Seo.MetaRobotsNoIndex = true; // Always block Robots from indexing this View
}

These set values can easily be rendered as HTML-tags inside Views through provided HtmlHelper-extensions:

<head>
    @Html.SeoTitle()

    @Html.SeoLinkCanonical()
    @Html.SeoMetaDescription()
    @Html.SeoMetaKeywords()
    @Html.SeoMetaRobotsIndex()
</head>

See the README-file on GitHub for the latest detailed information about this ASP.NET MVC SEO-framework. Or try it out through Nuget by running Install-Package AspNetMvcSeo in your project. You can even follow the absolutely latest build on MyGet.

IIS URL Rewrite-Rules Skipping Files-types

IIS Welcome

If you need to put in a rule for the IIS URL Rewrite Module, but need the rule to skip some file-endings and/or targets that are directories or actual files on disk, this is the post for you.

Following some SEO best-practices that tells us to use trailing slashes on our URLs I used the IIS Manager and added the IIS URL Rewrite-module's built-in rule "Append or remove the trailing slash symbol", which creates the following rule:

 <rule name="Add trailing slash" stopProcessing="true">
  <match url="(.*[^/])$" />
  <conditions>
    <add input="{REQUEST_FILENAME}" matchType="IsFile" negate="true" />
    <add input="{REQUEST_FILENAME}" matchType="IsDirectory" negate="true" />
  </conditions>
  <action type="Redirect" redirectType="Permanent" url="{R:1}/" />
</rule>

This rule takes into account and doesn't apply the rule to files and directories that exists on disk. But there is a big problem with this generic rule.

If you are dynamically serving up files with extensions, then an URL like:

http://website.com/about.html

will become:

http://website.com/about.html/

Adding conditions for specific file-endings

To solve this you can add conditions for certain file-endings, like .html and .aspx:

<conditions>
  <!-- ... -->
  <add input="{REQUEST_FILENAME}" pattern="(.*?)\.html$" negate="true" />
  <add input="{REQUEST_FILENAME}" pattern="(.*?)\.aspx$" negate="true" />
</conditions>

Since the rules above already don't apply for files physically on disk, you don't need to add file-endings like .css, .png or .js.

Update: Match ANY file-ending

If you want to match just any file-ending at all, you use the following pattern:

<conditions>
  <!-- ... -->
  <add input="{URL}" pattern=".*/[^.]*\.[\d\w]+$" negate="true" />
</conditions>

Guidelines for URI Design

Jacob Gillespie has worked on a post concerning URL Guidelines, that underwent much revision and was posted as a guest post on CSS-Tricks named Guidelines for URI Design.

Clean URIs are one component of a clean website, and it is an important one. The majority of end-user access to the Internet involves a URI, and whether or not the user actually enters the URI, they are working with one nonetheless.

Here is an outtake of the general principles of the article:

  • A URI must represent an object, uniquely and permanently - The URI must be unique so that it is a one-to-one match – one URI per one data object.
  • Be as human-friendly as possible - URIs should be designed with the end user in mind. SEO and ease of development should come second.
  • Consistency - URIs across a site must be consistent in format. Once you pick your URI structure, be consistent and follow it!
  • “Hackable” URIs - Related to consistency, URIs should be structured so that they are intelligibly “hackable” or changeable.
  • Keywords - The URI should be composed of keywords that are important to the content of the page. So, if the URI is for a blog post that has a long title, only the words important to the content of the page should be in the URI.

When it comes to technical details, here are their concerned bullet-points:

  • No WWW - The www. should be dropped from the website URI, as it is unnecessary typing and violates the rules of being as human-friendly as possible and not including unnecessary information in the URI.
  • Format - Google News has some interesting requirements for webpages that want to be listed in the Google News results – Google requires at least a 3-digit unique number.
  • All lowercase - All characters must be lowercase. Attempting to describe a URI to someone when mixed case is involved is next to impossible.
  • URI identifiers should be made URI friendly - A URI might contain the title of a post, and that title might contain characters that are not URI-friendly. That post title must therefore be made URI friendly. [...] Spaces should be replaced with hyphens.

Scott Mitchell has written an article on 4GuysFromRolla.com about Techniques for Preventing Duplicate URLs in Your Website.

A key tenet of search engine optimization is URL normalization, or URL canonicalization. URL normalization is the process of eliminating duplicate URLs in your website. This article explores four different ways to implement URL normalization in your ASP.NET website.

The important subjects of this article are the following:

  • First Things First: Deciding on a Canonical URL Format - Before we examine techniques for normalizing URLs, and certainly before such techniques can be implemented, we must first decide on a canonical URL format.
  • URL Normalization Using Permanent Redirects - [...] when a search engine spider receives a 301 status it updates its index with the new URL. Therefore, if anytime a request comes in for a non-canonical URL we immediately issue a permanent redirect to the same page but use the canonical form then a search engine spider crawling our site will only maintain the canonical form in its index.
  • Issuing Permanent Redirects From ASP.NET - Every time an incoming request is handled by the ASP.NET engine, it raises the BeginRequest event. You can execute code in response to this event by creating an HTTP Module or by creating the Application_BeginRequest event handler in Global.asax.
  • Rewriting URLs Into Canonical Form Using IIS 7's URL Rewrite Module - Shortly after releasing IIS 7, Microsoft created and released a free URL Rewrite Module. The URL Rewrite Module makes it easy to define URL rewriting rules in your Web.config file.
  • Rewriting URLs Into Canonical Form Using ISAPI_Rewrite - Microsoft's URL Rewriter Module is a great choice if you are using IIS 7, but if you are using previous version of IIS you're out of luck.
  • Telling Search Engine Spiders Your Canonical Form In Markup - Consider a URL that may include querystring parameters that don't affect the content rendered on the page or only affect non-essential parts of the page.
    In the case of YouTube, all video pages specify a <link> element like so, regardless of whether the querystring includes just the videoId or the videoId and other parameters:
    <link rel="canonical" href="/watch?v=videoId">
    

ASP.NET WebForms SEO: Compressing View State

In my previous post about Moving View State to Bottom of the Form I touched on one way to do some Search Engine Optimization in ASP.NET WebForms. Another thing you can do is to compress large View States on a page.

All the code in this article should be put in a class inheriting from System.Web.UI.Page. Like a page's Code-behind file or your own PageBase-class.

First we need some common logic for both loading and saving a compressed View State, the name of the hidden field to use.

private const string ViewStateFieldName = "SEOVIEWSTATE";

We start with the loading-part, where you see the use of my implementation of the CompressionHelper-class, which simplifies compression. It also solves the issue with DeflateStream and GZipStream inflating already compressed data, which is resolved in .NET Framework 4.

protected override object LoadPageStateFromPersistenceMedium() {
    return LoadCompressedPageState();
}

private object LoadCompressedPageState() {
    string viewState = Request.Form[ViewStateFieldName];
    if(string.IsNullOrEmpty(viewState)) {
        return string.Empty;
    }

    byte[] bytes = Convert.FromBase64String(viewState.Substring(1));

    bool isCompressed = Convert.ToBoolean(Convert.ToInt32(viewState.Substring(0, 1)));
    if(isCompressed) {
        bytes = CompressionHelper.Decompress(bytes);
    }

    string decompressedBase64 = Convert.ToBase64String(bytes);

    ObjectStateFormatter formatter = new ObjectStateFormatter();
    return formatter.Deserialize(decompressedBase64);
}

We then move on to the implementation of saving the page's state.

protected override void SavePageStateToPersistenceMedium(object state) {
    SaveCompressedPageState(state);
}

private void SaveCompressedPageState(object state) {
    byte[] viewStateBytes;
    using(MemoryStream stream = new MemoryStream()) {
        ObjectStateFormatter formatter = new ObjectStateFormatter();
        formatter.Serialize(stream, state);
        viewStateBytes = stream.ToArray();
    }

    byte[] compressed;
    bool successfulCompress = CompressionHelper.TryCompress(viewStateBytes, out compressed);
    string compressedBase64 =
        Convert.ToInt32(successfulCompress) + Convert.ToBase64String(compressed);

    ClientScript.RegisterHiddenField(ViewStateFieldName, compressedBase64);
}

So the summary is that this will compress the View State of an ASP.NET WebForms page, if the View State is large enough. Otherwise it will just keep it the way it is.

ASP.NET WebForms SEO: Moving View State to Bottom of the Form

As we all know, SEO is very important for almost all sites and some of us are still stuck with sites based on ASP.NET WebForm. But there are still things you can do to optimize these sites.

Since Google and other search-engines only indexes X amount of KB of a page, you don't want half of that content to be meaningless View State (which you should TRULY understand by now).

One solution is to compress the View State, but in this article I will show you how to move it to the bottom of the Form-tag. To achieve the latter, I will use a HttpModule, which will intercept all requests to files served through ASP.NET in the IIS.

First we need to hook in to the BeginRequest-event to only intercept requests to .aspx-pages.

public class ViewStateSeoHttpModule : IHttpModule {
    public void Init(HttpApplication context) {
        context.BeginRequest += new EventHandler(BeginRequest);
    }

    private void BeginRequest(object sender, EventArgs e) {
        HttpApplication application = sender as HttpApplication;

        bool isAspNetPageRequest = GetIsAspNetPageRequest(application);
        if(isAspNetPageRequest) {
            application.Context.Response.Filter =
                new ViewStateSeoFilter(application.Context.Response.Filter);
        }
    }

    private bool GetIsAspNetPageRequest(HttpApplication application) {
        string requestInfo = application.Context.Request.Url.Segments.Last();
        bool isAspNetPageRequest = requestInfo.ToLowerInvariant().Contains(".aspx");
        return isAspNetPageRequest;
    }
    // [...]

What we do here is add a Reponse Filter which moves the View State to the bottom of the Form. I made this to an internal class called ViewStateSeoFilter.

internal class ViewStateSeoFilter : Stream {
    private string _topPageHiddenFields;
    private Stream _originalFilter;

    public ViewStateSeoFilter(Stream originalFilter) {
        _originalFilter = originalFilter;
    }

    public override bool CanRead { get { return true; } }
    public override bool CanSeek { get { return true; } }
    public override bool CanWrite { get { return true; } }
    public override long Length { get { return 0; } }
    public override long Position { get; set; }

    public override void Flush() {
        _originalFilter.Flush();
    }

    public override int Read(byte[] buffer, int offset, int count) {
        return _originalFilter.Read(buffer, offset, count);
    }

    public override long Seek(long offset, SeekOrigin origin) {
        return _originalFilter.Seek(offset, origin);
    }

    public override void SetLength(long value) {
        _originalFilter.SetLength(value);
    }
    public override void Close() {
        _originalFilter.Close();
    }

    public override void Write(byte[] buffer, int offset, int count) {
        byte[] data = new byte[count];
        Buffer.BlockCopy(buffer, offset, data, 0, count);
        string html = Encoding.Default.GetString(buffer);

        html = ExtractPageTopHiddenFields(html);
        html = InsertExtractedHiddenFields(html);

        byte[] outdata = Encoding.Default.GetBytes(html);
        _originalFilter.Write(outdata, 0, outdata.GetLength(0));
    }

    private string ExtractPageTopHiddenFields(string html) {
        int formStartIndex = html.IndexOf("<form");
        if(formStartIndex < 0) {
            return html;
        }

        int divStartOpenIndex = html.IndexOf("<div", formStartIndex);
        if(divStartOpenIndex < 0) {
            return html;
        }
        int divStartCloseIndex = html.IndexOf(">", divStartOpenIndex);
        int divStartLenght = divStartCloseIndex - divStartOpenIndex;
        int divEndOpenIndex = html.IndexOf("</div", divStartOpenIndex);
        int divEndCloseIndex = html.IndexOf(">", divEndOpenIndex);

        int divContentLength = divEndOpenIndex - divStartCloseIndex - 1;

        _topPageHiddenFields = html.Substring(divStartCloseIndex + 1, divContentLength);
        html = html.Remove(divStartOpenIndex, divEndCloseIndex - divStartOpenIndex + 1);
        return html;
    }

    private string InsertExtractedHiddenFields(string html) {
        if(string.IsNullOrEmpty(_topPageHiddenFields)) {
            return html;
        }

        int insertIndex = html.IndexOf("</form>");
        if(insertIndex > 0) {
            html = html.Insert(insertIndex, _topPageHiddenFields + Environment.NewLine);
            _topPageHiddenFields = null;
        }

        return html;
    }
}

One bonus is that it moves all available Hidden Input HTML-tags controlled by ASP.NET to the bottom of the Form-tag. It seems to skip the Hidden Input that handles EventValidation, which is a good thing, because ASP.NET seems to want this tag early in the Page Life Cycle.