文档建议反馈控制台
首页
学习
活动
专区
圈层
工具
MCP广场
文章/答案/技术大牛
发布
社区首页 >问答首页 >如何在PHP中防止Curl请求时服务器过载

问如何在PHP中防止Curl请求时服务器过载
EN

Stack Overflow用户
提问于 2012-11-20 03:58:01
回答 1查看 2.6K关注 0票数 0

我正在向50个站点发送Curl请求，以使用Crons作业检查它们的运行状态。然而，我每次都能得到前10-15个站点的响应。我认为在此之后，服务器可能会超载并停止执行。

示例代码：

代码语言：javascript
复制
      foreach ($sites as $site) {
       if(Visit("http://www.domain.com")) {
           //saving in database
        }
       }

function Visit($url)

 {

$agent = "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)";$ch=curl_init();

 curl_setopt ($ch, CURLOPT_URL,$url );

 curl_setopt($ch, CURLOPT_USERAGENT, $agent);

 curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);

 curl_setopt ($ch,CURLOPT_VERBOSE,false);

 curl_setopt($ch, CURLOPT_TIMEOUT, 5);

 $page=curl_exec($ch);


  $httpcode = curl_getinfo($ch, CURLINFO_HTTP_CODE);

  curl_close($ch);

  if($httpcode>=200 && $httpcode<300) return true;

    else return false;

  }

请建议我一些技术，以防止服务器过载和管理，以获得所有网站的响应。

php
curl
EN

回答 1

Stack Overflow用户

回答已采纳

发布于 2012-11-20 04:28:37

您可以尝试使用以下curl_multi_exec，只需使用15.519232988358即可检查100个不同域名

代码语言：javascript
复制
$url = "google.com
facebook.com
youtube.com
yahoo.com
baidu.com
wikipedia.org
live.com
twitter.com
qq.com
amazon.com
taobao.com
linkedin.com
blogspot.com
google.co.in
yahoo.co.jp
sina.com.cn
google.de
yandex.ru
msn.com
wordpress.com
google.co.jp
google.com.hk
bing.com
ebay.com
google.co.uk
google.fr
vk.com
microsoft.com
babylon.com
weibo.com
googleusercontent.com
163.com
tumblr.com
apple.com
mail.ru
pinterest.com
soso.com
google.com.br
tmall.com
google.es
paypal.com
google.ru
google.it
xhamster.com
craigslist.org
sohu.com
blogger.com
fc2.com
imdb.com
go.com
ebay.de
google.co.id
zedo.com
alibaba.com
mywebsearch.com
google.com.tr
adf.ly
stackoverflow.com
blogspot.in
redtube.com
amazon.co.uk
360buy.com
google.com.au
alipay.com
sogou.com
about.com
instagram.com
ebay.co.uk
nytimes.com
livedoor.com
google.pl
netflix.com
imgur.com
uol.com.br
dailymotion.com
wordpress.org
360.cn
cnet.com
godaddy.com
youporn.com
bp.blogspot.com
ameblo.jp
mediafire.com
adcash.com
globo.com
chinaz.com
weather.com
incredibar.com
neobux.com
xnxx.com
google.nl
ehow.com
douban.com
google.com.sa
4shared.com
vimeo.com
livejournal.com
dropbox.com
renren.com
doubleclick.com";

echo "<pre>";
$url = explode("\n", $url);
set_time_limit(0);

$start = microtime(true);
multiplePost($url);
echo PHP_EOL, microtime(true) - $start;

输出

代码语言：javascript
复制
Status 301  for : http://www.google.com/ Error (x)
Status 302  for : https://facebook.com/ Error (x)
Status ok for : http://www.youtube.com/ Done (^)
Status ok for : http://www.yahoo.com/ Done (^)
Status ok for : http://baidu.com Done (^)
Status ok for : http://www.wikipedia.org/ Done (^)
Status 301  for : https://home.live.com/ Error (x)
Status ok for : http://twitter.com Done (^)
Status ok for : http://www.qq.com/ Done (^)
Status ok for : http://www.amazon.com/ Done (^)
Status ok for : http://www.taobao.com/ Done (^)
Status ok for : http://www.linkedin.com/ Done (^)
Status 302  for : https://accounts.google.com/ServiceLogin?service=blogger&passive=1209600&continue=http://www.blogger.com/home&followup=http://www.blogger.com/home<mpl=start Error (x)
Status ok for : http://www.google.co.in/ Done (^)
Status ok for : http://www.yahoo.co.jp/ Done (^)
Status 0  for : http://sina.com.cn Error (x)
Status ok for : http://www.google.de/ Done (^)
Status 301  for : http://www.yandex.ru/ Error (x)
Status ok for : http://www.msn.com/ Done (^)
Status ok for : http://wordpress.com Done (^)
Status ok for : http://www.google.co.jp/ Done (^)
Status ok for : http://www.google.com.hk/ Done (^)
Status ok for : http://www.bing.com/ Done (^)
Status 301  for : http://www.ebay.com Error (x)
Status ok for : http://www.google.co.uk/ Done (^)
Status ok for : http://www.google.fr/ Done (^)
Status ok for : http://vk.com Done (^)
Status ok for : http://www.microsoft.com/en-ng/default.aspx Done (^)
Status ok for : http://www.babylon.com/ Done (^)
Status ok for : http://weibo.com Done (^)
Status 0  for : http://googleusercontent.com Error (x)
Status 0  for : http://163.com Error (x)
Status 302  for : https://www.tumblr.com/ Error (x)
Status ok for : http://www.apple.com/ Done (^)
Status ok for : http://mail.ru Done (^)
Status ok for : http://pinterest.com Done (^)
Status 0  for : http://soso.com Error (x)
Status ok for : http://www.google.com.br/ Done (^)
Status ok for : http://www.tmall.com/ Done (^)
Status ok for : http://www.google.es/ Done (^)
Status 302  for : https://paypal.com/ Error (x)
Status ok for : http://www.google.ru/ Done (^)
Status ok for : http://www.google.it/ Done (^)
Status ok for : http://xhamster.com Done (^)
Status ok for : http://www.craigslist.org/about/sites/ Done (^)
Status 302  for : http://www.sohu.com/ Error (x)
Status 302  for : https://accounts.google.com/ServiceLogin?service=blogger&passive=1209600&continue=http://www.blogger.com/home&followup=http://www.blogger.com/home<mpl=start Error (x)
Status ok for : http://fc2.com Done (^)
Status ok for : http://www.imdb.com/ Done (^)
Status ok for : http://go.com Done (^)
Status 301  for : http://www.ebay.de Error (x)
Status ok for : http://www.google.co.id/ Done (^)
Status ok for : http://www.zedo.com/ Done (^)
Status ok for : http://www.alibaba.com/ Done (^)
Status ok for : http://home.mywebsearch.com/ Done (^)
Status ok for : http://www.google.com.tr/ Done (^)
Status ok for : http://adf.ly Done (^)
Status ok for : http://stackoverflow.com Done (^)
Status 302  for : http://www.google.com/ Error (x)
Status ok for : http://www.redtube.com/ Done (^)
Status ok for : http://www.amazon.co.uk/ Done (^)
Status ok for : http://360buy.com Done (^)
Status ok for : http://www.google.com.au/ Done (^)
Status 301  for : https://www.alipay.com/?src=alipay.com Error (x)
Status ok for : http://www.sogou.com/ Done (^)
Status ok for : http://www.about.com/ Done (^)
Status ok for : http://instagram.com Done (^)
Status 301  for : http://www.ebay.co.uk Error (x)
Status ok for : http://www.nytimes.com/ Done (^)
Status ok for : http://www.livedoor.com/ Done (^)
Status ok for : http://www.google.pl/ Done (^)
Status 301  for : http://www.netflix.com/ Error (x)
Status ok for : http://imgur.com Done (^)
Status ok for : http://www.uol.com.br/ Done (^)
Status 301  for : http://www.dailymotion.com/ Error (x)
Status ok for : http://wordpress.org Done (^)
Status ok for : http://360.cn Done (^)
Status ok for : http://www.cnet.com/ Done (^)
Status ok for : http://www.godaddy.com/ Done (^)
Status ok for : http://www.youporn.com/ Done (^)
Status 0  for : http://bp.blogspot.com Error (x)
Status ok for : http://ameblo.jp Done (^)
Status ok for : http://www.mediafire.com/ Done (^)
Status 301  for : https://www.adcash.com/index.php Error (x)
Status 301  for : http://www.globo.com/ Error (x)
Status ok for : http://chinaz.com Done (^)
Status ok for : http://www.weather.com/ Done (^)
Status ok for : http://incredibar.com/essentials/homepage Done (^)
Status ok for : http://www.neobux.com/ Done (^)
Status 301  for : http://www.xnxx.com/ Error (x)
Status ok for : http://www.google.nl/ Done (^)
Status ok for : http://www.ehow.com/ Done (^)
Status 0  for : http://douban.com Error (x)
Status ok for : http://www.google.com.sa/ Done (^)
Status 301  for : http://www.4shared.com Error (x)
Status ok for : http://vimeo.com Done (^)
Status ok for : http://www.livejournal.com/ Done (^)
Status 302  for : https://www.dropbox.com/ Error (x)
Status ok for : http://renren.com Done (^)
Status ok for : http://www.google.com/doubleclick/ Done (^)

15.519232988358 <--------------- Total Time Taken

使用的函数

代码语言：javascript
复制
function multiplePost($nodes) {
    $mh = curl_multi_init();
    $curl_array = array();
    foreach ( $nodes as $i => $domain ) {
        $domain = trim($domain);
        $curl_array[$i] = curl_init("http://$domain");
        curl_setopt($curl_array[$i], CURLOPT_RETURNTRANSFER, true);
        curl_setopt($curl_array[$i], CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 (.NET CLR 3.5.30729)');
        curl_setopt($curl_array[$i], CURLOPT_CONNECTTIMEOUT, 5);
        curl_setopt($curl_array[$i], CURLOPT_TIMEOUT, 15);
        curl_setopt($curl_array[$i], CURLOPT_FOLLOWLOCATION, true);
        curl_multi_add_handle($mh, $curl_array[$i]);
    }

    echo PHP_EOL, PHP_EOL;

    $running = NULL;
    do {
        usleep(10000);
        curl_multi_exec($mh, $running);
    } while ( $running > 0 );
    $res = array();
    foreach ( $nodes as $i => $url ) {

        $curlErrorCode = curl_errno($curl_array[$i]);

        if ($curlErrorCode === 0) {
            $info = curl_getinfo($curl_array[$i]);
            $info['url'] = trim($info['url']);
            if ($info['http_code'] == 200) {
                echo "Status ok for : {$info['url']} Done (^)", PHP_EOL;
                $content = curl_multi_getcontent($curl_array[$i]); // get content
            } else {
                echo "Status {$info['http_code'] }  for : {$info['url']} Error (x)", PHP_EOL;
            }
        }
        curl_multi_remove_handle($mh, $curl_array[$i]);
        curl_close($curl_array[$i]);
        flush();
        ob_flush();
    }
    curl_multi_close($mh);
}
票数 7
EN
页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持
原文链接：

https://stackoverflow.com/questions/13461194

复制
相关文章

相似问题

领券

腾讯云开发者

扫码关注腾讯云开发者

扫码关注腾讯云开发者

领取腾讯云代金券

热门产品

热门推荐

更多推荐

Copyright © 2013 - 2026 Tencent Cloud. All Rights Reserved. 腾讯云 版权所有 

深圳市腾讯计算机系统有限公司 ICP备案/许可证号：粤B2-20090059 粤公网安备44030502008569号

腾讯云计算（北京）有限责任公司 京ICP证150476号 |  京ICP备11018762号

问题归档专栏文章快讯文章归档关键词归档开发者手册归档开发者手册 Section 归档