How to create a cross domain HTTP request

Question

How to create a cross domain HTTP request

699 Views Asked by Midnight_Blaze At 19 February 2017 at 10:51

I have a website, and I need a way to get html data from a different website via an http request, and I've looked around for ways to implement it and most say via an ajax call instead.

An ajax call is blocked by linked in so I want to try a plain cross domain http request and hope it's not blocked one way or another.

Original Q&A

There are 1 best solutions below

**rene** · Accepted Answer · 2017-02-19T13:31:46.930000

If you have a server running and are able to run code on it, you can make the HTTP call server side. Keep in mind though that most sites only allow so many calls per IP address so you can't serve a lot of users this way.

This is a simple httpListener that downloads an websites content when the QueryString contains ?site=http://linkedin.com:

// setup an listener
using(var listener = new HttpListener())
{
    // on port 8080
    listener.Prefixes.Add("http://+:8080/");
    listener.Start();
    while(true) 
    {
        // wait for a connect
        var ctx = listener.GetContext();
        var req = ctx.Request;
        var resp = ctx.Response;
        // default page 
        var cnt = "<html><body><a href=\"/?site=http://linkedin.com\">click me</a> </body></html>";
        foreach(var key in req.QueryString.Keys)
        {
          if (key!=null) 
          {
             // if the url contains ?site=some url to an site
            switch(key.ToString()) 
            {
                case "site":
                // lets download
                var wc = new WebClient();
                // store html in cnt
                cnt = wc.DownloadString(req.QueryString[key.ToString()]);
                // when needed you can do caching or processing here
                // of the results, depending on your needs
                break;
                default:
                break;
            }
          }
        }
        // output whatever is in cnt to the calling browser
        using(var sw = new StreamWriter(resp.OutputStream))
        {
            sw.Write(cnt);
        }
    }
}

To make above code work you might have to set permissions for the url, if you'r on your development box do:

netsh http add urlacl url=http://+:8080/ user=Everyone   listen=yes

On production use sane values for the user.

Once that is set run the above code and point your browser to

 http://localhost:8080/

(notice the / at the end)

You'll get a simple page with a link on it:

click me

Clicking that link will send a new request to the httplistener but this time with the query string site=http://linkedin.com. The server side code will fetch the http content that is at the url given, in this case from LinkedIn.com. The result is send back one-on-one to the browser but you can do post-processing/caching etc, depending on your requirements.

Legal notice/disclaimer

Most sites don't like being scraped this way and their Terms of Service might actually forbid it. Make sure you don't do illegal things that either harms site reliability or leads to legal actions against you.

How to create a cross domain HTTP request

There are 1 best solutions below

Legal notice/disclaimer

Related Questions in JAVASCRIPT

Related Questions in C#

Related Questions in HTTP-REQUEST

Trending Questions

Popular # Hahtags

Popular Questions