cURL / PHP cant fetch this - but all browser can?

290 Views Asked by Morten Pedersen At 15 March 2018 at 23:16

I cant acces this site with any of Bellow methods, the $url adress Works in all my browsers but, i just cant fetch data from that site.... how is that possible ? Not even the robots.txt ('https://www.natterer-modellbau.de/robots.txt) cant be fetched, other than in a browser....

I see results on Google from that website, how can Google acces the site when I cant ?

The page has rejected my crawler from first try, så they cant have bloked my servers IP allready ? and my script can acces all other URL`s -- im frustrated :) please help...

$url = 'https://www.natterer-modellbau.de/Flugzeuge';

$pageHeaders = get_headers($url,1); // DOES NOT WORK - TIMES OUT
file_get_contents($url); // DOES NOT WORK -Times OUT 

$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1");
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
$res = curl_exec($ch);
$rescode = curl_getinfo($ch, CURLINFO_HTTP_CODE); 
curl_close($ch) ;
echo $res; // DOES NOT WORK - TIMES OUT

Original Q&A

There are 1 best solutions below

Ari Singh On 15 March 2018 at 23:29

Works for me. You may be behind a proxy.

php > $url = 'https://www.natterer-modellbau.de/Flugzeuge';
php >
php > $pageHeaders = get_headers($url,1); // DOES NOT WORK - TIMES OUT
php > file_get_contents($url); // DOES NOT WORK -Times OUT
php >
php > $ch = curl_init();
php > curl_setopt($ch, CURLOPT_URL, $url);
php > curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
php > curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1");
php > curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
php > $res = curl_exec($ch);
php > $rescode = curl_getinfo($ch, CURLINFO_HTTP_CODE);
php > curl_close($ch) ;
php > echo $res;
<!DOCTYPE html>
<html lang="de">
<head>

        <meta http-equiv="content-type" content="text/html; charset=iso-8859-1">
        <meta name="description" content="Elektro, Segler/E-Segler, Verbrenner">
        <meta name="keywords" content="Elektro, Segler/E-Segler, Verbrenner">

.... Many lines omitted

finally end of html

        </script>

    <script>
        jtl.load(["asset/plugin_js_head?v=4.05","asset/jtl3.js?v=4.05","asset/plugin_js_body?v=4.05",]);
            </script>

</body>
</html>
php >

cURL / PHP cant fetch this - but all browser can?

There are 1 best solutions below

Related Questions in PHP

Related Questions in CURL

Related Questions in BROWSER

Related Questions in FILE-GET-CONTENTS

Related Questions in GET-HEADERS

Trending Questions

Popular # Hahtags

Popular Questions