C语言libcurl例程：multi 多线程，多任务
2024-04-12 16:42:05
目录
A multi-threaded example that uses pthreads to fetch several files at once
multi_socket API using libuv
using the multi interface to do a single download
using the multi interface to do a multipart formpost without blocking
single download with the multi interface's curl_multi_poll
using the multi interface to do a multipart formpost without blocking
multi_socket API using libevent
multi interface code doing two parallel HTTP transfers
multi interface and debug callback
A basic application source code using the multi interface doing two transfers in parallel.
A multi-threaded example that uses pthreads to fetch several files at once
/**************************************************************************** _ _ ____ _* Project ___| | | | _ \| |* / __| | | | |_) | |* | (__| |_| | _ <| |___* \___|\___/|_| \_\_____|** Copyright (C) 1998 - 2019, Daniel Stenberg, <daniel@haxx.se>, et al.** This software is licensed as described in the file COPYING, which* you should have received as part of this distribution. The terms* are also available at https://curl.haxx.se/docs/copyright.html.** You may opt to use, copy, modify, merge, publish, distribute and/or sell* copies of the Software, and permit persons to whom the Software is* furnished to do so, under the terms of the COPYING file.** This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY* KIND, either express or implied.****************************************************************************/
/* <DESC>* A multi-threaded example that uses pthreads to fetch several files at once* </DESC>*/#include <stdio.h>
#include <pthread.h>
#include <curl/curl.h>#define NUMT 4/*List of URLs to fetch.If you intend to use a SSL-based protocol here you might need to setup TLSlibrary mutex callbacks as described here:https://curl.haxx.se/libcurl/c/threadsafe.html*/
const char * const urls[NUMT]= {
// "https://curl.haxx.se/",
// "ftp://cool.haxx.se/",
// "https://www.cag.se/",
// "www.haxx.se""http://10.170.6.66","http://localhost","http://10.170.6.66/crtl-docs/","http://10.170.6.66/crtl-docs/linux-kernel-map/index.html"
};static void *pull_one_url(void *url)
{CURL *curl;curl = curl_easy_init();curl_easy_setopt(curl, CURLOPT_URL, url);curl_easy_perform(curl); /* ignores error */curl_easy_cleanup(curl);return NULL;
}/*int pthread_create(pthread_t *new_thread_ID,const pthread_attr_t *attr,void * (*start_func)(void *), void *arg);
*/int main(int argc, char **argv)
{pthread_t tid[NUMT];int i;/* Must initialize libcurl before any threads are started */curl_global_init(CURL_GLOBAL_ALL);for(i = 0; i< NUMT; i++) {int error = pthread_create(&tid[i],NULL, /* default attributes please */pull_one_url,(void *)urls[i]);if(0 != error)fprintf(stderr, "Couldn't run thread number %d, errno %d\n", i, error);elsefprintf(stderr, "Thread %d, gets %s\n", i, urls[i]);}/* now wait for all threads to terminate */for(i = 0; i< NUMT; i++) {pthread_join(tid[i], NULL);fprintf(stderr, "Thread %d terminated\n", i);}return 0;
}
multi_socket API using libuv
https://rtoax.blog.csdn.net/article/details/105711077#%E6%96%87%E4%BB%B6I%20%2F%C2%A0
/**************************************************************************** _ _ ____ _* Project ___| | | | _ \| |* / __| | | | |_) | |* | (__| |_| | _ <| |___* \___|\___/|_| \_\_____|** Copyright (C) 1998 - 2018, Daniel Stenberg, <daniel@haxx.se>, et al.** This software is licensed as described in the file COPYING, which* you should have received as part of this distribution. The terms* are also available at https://curl.haxx.se/docs/copyright.html.** You may opt to use, copy, modify, merge, publish, distribute and/or sell* copies of the Software, and permit persons to whom the Software is* furnished to do so, under the terms of the COPYING file.** This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY* KIND, either express or implied.****************************************************************************//* <DESC>* multi_socket API using libuv* </DESC>*/
/* Example application using the multi socket interface to download multiplefiles in parallel, powered by libuv.Requires libuv and (of course) libcurl.See https://nikhilm.github.io/uvbook/ for more information on libuv.
*/#include <stdio.h>
#include <stdlib.h>
#include <uv.h>
#include <curl/curl.h>uv_loop_t *loop;
CURLM *curl_handle;
uv_timer_t timeout;typedef struct curl_context_s {uv_poll_t poll_handle;curl_socket_t sockfd;
} curl_context_t;static curl_context_t* create_curl_context(curl_socket_t sockfd)
{curl_context_t *context;context = (curl_context_t *) malloc(sizeof(*context));context->sockfd = sockfd;uv_poll_init_socket(loop, &context->poll_handle, sockfd);context->poll_handle.data = context;return context;
}static void curl_close_cb(uv_handle_t *handle)
{curl_context_t *context = (curl_context_t *) handle->data;free(context);
}static void destroy_curl_context(curl_context_t *context)
{uv_close((uv_handle_t *) &context->poll_handle, curl_close_cb);
}static void add_download(const char *url, int num)
{char filename[50];FILE *file;CURL *handle;snprintf(filename, 50, "%d.download", num);file = fopen(filename, "wb");if(!file) {fprintf(stderr, "Error opening %s\n", filename);return;}handle = curl_easy_init();curl_easy_setopt(handle, CURLOPT_WRITEDATA, file);curl_easy_setopt(handle, CURLOPT_PRIVATE, file);curl_easy_setopt(handle, CURLOPT_URL, url);curl_multi_add_handle(curl_handle, handle);fprintf(stderr, "Added download %s -> %s\n", url, filename);
}static void check_multi_info(void)
{char *done_url;CURLMsg *message;int pending;CURL *easy_handle;FILE *file;while((message = curl_multi_info_read(curl_handle, &pending))) {switch(message->msg) {case CURLMSG_DONE:/* Do not use message data after calling curl_multi_remove_handle() andcurl_easy_cleanup(). As per curl_multi_info_read() docs:"WARNING: The data the returned pointer points to will not survivecalling curl_multi_cleanup, curl_multi_remove_handle orcurl_easy_cleanup." */easy_handle = message->easy_handle;curl_easy_getinfo(easy_handle, CURLINFO_EFFECTIVE_URL, &done_url);curl_easy_getinfo(easy_handle, CURLINFO_PRIVATE, &file);printf("%s DONE\n", done_url);curl_multi_remove_handle(curl_handle, easy_handle);curl_easy_cleanup(easy_handle);if(file) {fclose(file);}break;default:fprintf(stderr, "CURLMSG default\n");break;}}
}static void curl_perform(uv_poll_t *req, int status, int events)
{int running_handles;int flags = 0;curl_context_t *context;if(events & UV_READABLE)flags |= CURL_CSELECT_IN;if(events & UV_WRITABLE)flags |= CURL_CSELECT_OUT;context = (curl_context_t *) req->data;curl_multi_socket_action(curl_handle, context->sockfd, flags,&running_handles);check_multi_info();
}static void on_timeout(uv_timer_t *req)
{int running_handles;curl_multi_socket_action(curl_handle, CURL_SOCKET_TIMEOUT, 0,&running_handles);check_multi_info();
}static int start_timeout(CURLM *multi, long timeout_ms, void *userp)
{if(timeout_ms < 0) {uv_timer_stop(&timeout);}else {if(timeout_ms == 0)timeout_ms = 1; /* 0 means directly call socket_action, but we'll do itin a bit */uv_timer_start(&timeout, on_timeout, timeout_ms, 0);}return 0;
}static int handle_socket(CURL *easy, curl_socket_t s, int action, void *userp,void *socketp)
{curl_context_t *curl_context;int events = 0;switch(action) {case CURL_POLL_IN:case CURL_POLL_OUT:case CURL_POLL_INOUT:curl_context = socketp ?(curl_context_t *) socketp : create_curl_context(s);curl_multi_assign(curl_handle, s, (void *) curl_context);if(action != CURL_POLL_IN)events |= UV_WRITABLE;if(action != CURL_POLL_OUT)events |= UV_READABLE;uv_poll_start(&curl_context->poll_handle, events, curl_perform);break;case CURL_POLL_REMOVE:if(socketp) {uv_poll_stop(&((curl_context_t*)socketp)->poll_handle);destroy_curl_context((curl_context_t*) socketp);curl_multi_assign(curl_handle, s, NULL);}break;default:abort();}return 0;
}int main(int argc, char **argv)
{loop = uv_default_loop();if(argc <= 1)return 0;if(curl_global_init(CURL_GLOBAL_ALL)) {fprintf(stderr, "Could not init curl\n");return 1;}uv_timer_init(loop, &timeout);curl_handle = curl_multi_init();curl_multi_setopt(curl_handle, CURLMOPT_SOCKETFUNCTION, handle_socket);curl_multi_setopt(curl_handle, CURLMOPT_TIMERFUNCTION, start_timeout);while(argc-- > 1) {add_download(argv[argc], argc);}uv_run(loop, UV_RUN_DEFAULT);curl_multi_cleanup(curl_handle);return 0;
}
using the multi interface to do a single download
/**************************************************************************** _ _ ____ _* Project ___| | | | _ \| |* / __| | | | |_) | |* | (__| |_| | _ <| |___* \___|\___/|_| \_\_____|** Copyright (C) 1998 - 2018, Daniel Stenberg, <daniel@haxx.se>, et al.** This software is licensed as described in the file COPYING, which* you should have received as part of this distribution. The terms* are also available at https://curl.haxx.se/docs/copyright.html.** You may opt to use, copy, modify, merge, publish, distribute and/or sell* copies of the Software, and permit persons to whom the Software is* furnished to do so, under the terms of the COPYING file.** This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY* KIND, either express or implied.****************************************************************************/
/* <DESC>* using the multi interface to do a single download* </DESC>*/#include <stdio.h>
#include <string.h>/* somewhat unix-specific */
#include <sys/time.h>
#include <unistd.h>/* curl stuff */
#include <curl/curl.h>#ifdef _WIN32
#define WAITMS(x) Sleep(x)
#else
/* Portable sleep for platforms other than Windows. */
#define WAITMS(x) \struct timeval wait = { 0, (x) * 1000 }; \(void)select(0, NULL, NULL, NULL, &wait);
#endif/** Simply download a HTTP file.*/
int main(void)
{CURL *http_handle;CURLM *multi_handle;int still_running = 0; /* keep number of running handles */int repeats = 0;curl_global_init(CURL_GLOBAL_DEFAULT);http_handle = curl_easy_init();/* set the options (I left out a few, you'll get the point anyway) */curl_easy_setopt(http_handle, CURLOPT_URL, "https://10.170.6.66/");/* init a multi stack */multi_handle = curl_multi_init();/* add the individual transfers */curl_multi_add_handle(multi_handle, http_handle);/* we start some action by calling perform right away */curl_multi_perform(multi_handle, &still_running);while(still_running) {CURLMcode mc; /* curl_multi_wait() return code */int numfds;/* wait for activity, timeout or "nothing" */mc = curl_multi_wait(multi_handle, NULL, 0, 1000, &numfds);if(mc != CURLM_OK) {fprintf(stderr, "curl_multi_wait() failed, code %d.\n", mc);break;}/* 'numfds' being zero means either a timeout or no file descriptors towait for. Try timeout on first occurrence, then assume no filedescriptors and no file descriptors to wait for means wait for 100milliseconds. */if(!numfds) {repeats++; /* count number of repeated zero numfds */if(repeats > 1) {WAITMS(100); /* sleep 100 milliseconds */}}elserepeats = 0;curl_multi_perform(multi_handle, &still_running);}curl_multi_remove_handle(multi_handle, http_handle);curl_easy_cleanup(http_handle);curl_multi_cleanup(multi_handle);curl_global_cleanup();return 0;
}
using the multi interface to do a multipart formpost without blocking
/**************************************************************************** _ _ ____ _* Project ___| | | | _ \| |* / __| | | | |_) | |* | (__| |_| | _ <| |___* \___|\___/|_| \_\_____|** Copyright (C) 1998 - 2018, Daniel Stenberg, <daniel@haxx.se>, et al.** This software is licensed as described in the file COPYING, which* you should have received as part of this distribution. The terms* are also available at https://curl.haxx.se/docs/copyright.html.** You may opt to use, copy, modify, merge, publish, distribute and/or sell* copies of the Software, and permit persons to whom the Software is* furnished to do so, under the terms of the COPYING file.** This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY* KIND, either express or implied.****************************************************************************/
/* <DESC>* using the multi interface to do a multipart formpost without blocking* </DESC>*/#include <stdio.h>
#include <string.h>
#include <sys/time.h>#include <curl/curl.h>int main(void)
{CURL *curl;CURLM *multi_handle;int still_running = 0;curl_mime *form = NULL;curl_mimepart *field = NULL;struct curl_slist *headerlist = NULL;static const char buf[] = "Expect:";curl = curl_easy_init();multi_handle = curl_multi_init();if(curl && multi_handle) {/* Create the form */form = curl_mime_init(curl);/* Fill in the file upload field */field = curl_mime_addpart(form);curl_mime_name(field, "sendfile");curl_mime_filedata(field, "multi-post.c");/* Fill in the filename field */field = curl_mime_addpart(form);curl_mime_name(field, "filename");curl_mime_data(field, "multi-post.c", CURL_ZERO_TERMINATED);/* Fill in the submit field too, even if this is rarely needed */field = curl_mime_addpart(form);curl_mime_name(field, "submit");curl_mime_data(field, "send", CURL_ZERO_TERMINATED);/* initialize custom header list (stating that Expect: 100-continue is notwanted */headerlist = curl_slist_append(headerlist, buf);/* what URL that receives this POST */curl_easy_setopt(curl, CURLOPT_URL, "https://www.example.com/upload.cgi");//CGI: 公共网关接口（Common Gateway Interface，CGI）是Web 服务器运行时外部程序的规范，按CGI 编写的程序可以扩展服务器功能。curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L);curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headerlist);curl_easy_setopt(curl, CURLOPT_MIMEPOST, form);curl_multi_add_handle(multi_handle, curl);curl_multi_perform(multi_handle, &still_running);while(still_running) {struct timeval timeout;int rc; /* select() return code */CURLMcode mc; /* curl_multi_fdset() return code */fd_set fdread;fd_set fdwrite;fd_set fdexcep;int maxfd = -1;long curl_timeo = -1;FD_ZERO(&fdread);FD_ZERO(&fdwrite);FD_ZERO(&fdexcep);/* set a suitable timeout to play around with */timeout.tv_sec = 1;timeout.tv_usec = 0;curl_multi_timeout(multi_handle, &curl_timeo);if(curl_timeo >= 0) {timeout.tv_sec = curl_timeo / 1000;if(timeout.tv_sec > 1)timeout.tv_sec = 1;elsetimeout.tv_usec = (curl_timeo % 1000) * 1000;}/* get file descriptors from the transfers */mc = curl_multi_fdset(multi_handle, &fdread, &fdwrite, &fdexcep, &maxfd);if(mc != CURLM_OK) {fprintf(stderr, "curl_multi_fdset() failed, code %d.\n", mc);break;}/* On success the value of maxfd is guaranteed to be >= -1. We callselect(maxfd + 1, ...); specially in case of (maxfd == -1) there areno fds ready yet so we call select(0, ...) --or Sleep() on Windows--to sleep 100ms, which is the minimum suggested value in thecurl_multi_fdset() doc. */if(maxfd == -1) {
#ifdef _WIN32Sleep(100);rc = 0;
#else/* Portable sleep for platforms other than Windows. */struct timeval wait = { 0, 100 * 1000 }; /* 100ms */rc = select(0, NULL, NULL, NULL, &wait);
#endif}else {/* Note that on some platforms 'timeout' may be modified by select().If you need access to the original value save a copy beforehand. */rc = select(maxfd + 1, &fdread, &fdwrite, &fdexcep, &timeout);}switch(rc) {case -1:/* select error */break;case 0:default:/* timeout or readable/writable sockets */printf("perform!\n");curl_multi_perform(multi_handle, &still_running);printf("running: %d!\n", still_running);break;}}curl_multi_cleanup(multi_handle);/* always cleanup */curl_easy_cleanup(curl);/* then cleanup the form */curl_mime_free(form);/* free slist */curl_slist_free_all(headerlist);}return 0;
}
single download with the multi interface's curl_multi_poll
/**************************************************************************** _ _ ____ _* Project ___| | | | _ \| |* / __| | | | |_) | |* | (__| |_| | _ <| |___* \___|\___/|_| \_\_____|** Copyright (C) 1998 - 2019, Daniel Stenberg, <daniel@haxx.se>, et al.** This software is licensed as described in the file COPYING, which* you should have received as part of this distribution. The terms* are also available at https://curl.haxx.se/docs/copyright.html.** You may opt to use, copy, modify, merge, publish, distribute and/or sell* copies of the Software, and permit persons to whom the Software is* furnished to do so, under the terms of the COPYING file.** This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY* KIND, either express or implied.****************************************************************************/
/* <DESC>* single download with the multi interface's curl_multi_poll* </DESC>*/#include <stdio.h>
#include <string.h>/* somewhat unix-specific */
#include <sys/time.h>
#include <unistd.h>/* curl stuff */
#include <curl/curl.h>int main(void)
{CURL *http_handle;CURLM *multi_handle;int still_running = 1; /* keep number of running handles */curl_global_init(CURL_GLOBAL_DEFAULT);http_handle = curl_easy_init();curl_easy_setopt(http_handle, CURLOPT_URL, "https://10.170.6.66/");multi_handle = curl_multi_init();curl_multi_add_handle(multi_handle, http_handle);while(still_running) {CURLMcode mc; /* curl_multi_poll() return code */int numfds;/* we start some action by calling perform right away */mc = curl_multi_perform(multi_handle, &still_running);if(still_running)/* wait for activity, timeout or "nothing" */mc = curl_multi_poll(multi_handle, NULL, 0, 1000, &numfds);if(mc != CURLM_OK) {fprintf(stderr, "curl_multi_wait() failed, code %d.\n", mc);break;}}curl_multi_remove_handle(multi_handle, http_handle);curl_easy_cleanup(http_handle);curl_multi_cleanup(multi_handle);curl_global_cleanup();return 0;
}
using the multi interface to do a multipart formpost without blocking
/**************************************************************************** _ _ ____ _* Project ___| | | | _ \| |* / __| | | | |_) | |* | (__| |_| | _ <| |___* \___|\___/|_| \_\_____|** Copyright (C) 1998 - 2018, Daniel Stenberg, <daniel@haxx.se>, et al.** This software is licensed as described in the file COPYING, which* you should have received as part of this distribution. The terms* are also available at https://curl.haxx.se/docs/copyright.html.** You may opt to use, copy, modify, merge, publish, distribute and/or sell* copies of the Software, and permit persons to whom the Software is* furnished to do so, under the terms of the COPYING file.** This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY* KIND, either express or implied.****************************************************************************/
/* <DESC>* using the multi interface to do a multipart formpost without blocking* </DESC>*/#include <stdio.h>
#include <string.h>
#include <sys/time.h>#include <curl/curl.h>int main(void)
{CURL *curl;CURLM *multi_handle;int still_running = 0;struct curl_httppost *formpost = NULL;struct curl_httppost *lastptr = NULL;struct curl_slist *headerlist = NULL;static const char buf[] = "Expect:";/* Fill in the file upload field. This makes libcurl load data fromthe given file name when curl_easy_perform() is called. *///curl_formadd : 用于为multipart/formdata HTTP POST请求添加部分。curl_formadd(&formpost,&lastptr,CURLFORM_COPYNAME, "sendfile",CURLFORM_FILE, "postit2.c",CURLFORM_END);/* Fill in the filename field */curl_formadd(&formpost,&lastptr,CURLFORM_COPYNAME, "filename",CURLFORM_COPYCONTENTS, "postit2.c",CURLFORM_END);/* Fill in the submit field too, even if this is rarely needed */curl_formadd(&formpost,&lastptr,CURLFORM_COPYNAME, "submit",CURLFORM_COPYCONTENTS, "send",CURLFORM_END);curl = curl_easy_init();multi_handle = curl_multi_init();/* initialize custom header list (stating that Expect: 100-continue is notwanted */headerlist = curl_slist_append(headerlist, buf);if(curl && multi_handle) {/* what URL that receives this POST */curl_easy_setopt(curl, CURLOPT_URL, "https://10.170.6.66/study/");curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L);curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headerlist);curl_easy_setopt(curl, CURLOPT_HTTPPOST, formpost);curl_multi_add_handle(multi_handle, curl);curl_multi_perform(multi_handle, &still_running);while(still_running) {struct timeval timeout;int rc; /* select() return code */CURLMcode mc; /* curl_multi_fdset() return code */fd_set fdread;fd_set fdwrite;fd_set fdexcep;int maxfd = -1;long curl_timeo = -1;FD_ZERO(&fdread);FD_ZERO(&fdwrite);FD_ZERO(&fdexcep);/* set a suitable timeout to play around with */timeout.tv_sec = 1;timeout.tv_usec = 0;curl_multi_timeout(multi_handle, &curl_timeo);if(curl_timeo >= 0) {timeout.tv_sec = curl_timeo / 1000;if(timeout.tv_sec > 1)timeout.tv_sec = 1;elsetimeout.tv_usec = (curl_timeo % 1000) * 1000;}/* get file descriptors from the transfers */mc = curl_multi_fdset(multi_handle, &fdread, &fdwrite, &fdexcep, &maxfd);if(mc != CURLM_OK) {fprintf(stderr, "curl_multi_fdset() failed, code %d.\n", mc);break;}/* On success the value of maxfd is guaranteed to be >= -1. We callselect(maxfd + 1, ...); specially in case of (maxfd == -1) there areno fds ready yet so we call select(0, ...) --or Sleep() on Windows--to sleep 100ms, which is the minimum suggested value in thecurl_multi_fdset() doc. */if(maxfd == -1) {
#ifdef _WIN32Sleep(100);rc = 0;
#else/* Portable sleep for platforms other than Windows. */struct timeval wait = { 0, 100 * 1000 }; /* 100ms */rc = select(0, NULL, NULL, NULL, &wait);
#endif}else {/* Note that on some platforms 'timeout' may be modified by select().If you need access to the original value save a copy beforehand. */rc = select(maxfd + 1, &fdread, &fdwrite, &fdexcep, &timeout);}switch(rc) {case -1:/* select error */break;case 0:default:/* timeout or readable/writable sockets */printf("perform!\n");curl_multi_perform(multi_handle, &still_running);printf("running: %d!\n", still_running);break;}}curl_multi_cleanup(multi_handle);/* always cleanup */curl_easy_cleanup(curl);/* then cleanup the formpost chain */curl_formfree(formpost);/* free slist */curl_slist_free_all(headerlist);}return 0;
}
multi_socket API using libevent
/**************************************************************************** _ _ ____ _* Project ___| | | | _ \| |* / __| | | | |_) | |* | (__| |_| | _ <| |___* \___|\___/|_| \_\_____|** Copyright (C) 1998 - 2019, Daniel Stenberg, <daniel@haxx.se>, et al.** This software is licensed as described in the file COPYING, which* you should have received as part of this distribution. The terms* are also available at https://curl.haxx.se/docs/copyright.html.** You may opt to use, copy, modify, merge, publish, distribute and/or sell* copies of the Software, and permit persons to whom the Software is* furnished to do so, under the terms of the COPYING file.** This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY* KIND, either express or implied.****************************************************************************//* <DESC>* multi_socket API using libevent* </DESC>*/#include <stdio.h>
#include <stdlib.h>
#include <event2/event.h>
#include <curl/curl.h>struct event_base *base;
CURLM *curl_handle;
struct event *timeout;typedef struct curl_context_s {struct event *event;curl_socket_t sockfd;
} curl_context_t;static void curl_perform(int fd, short event, void *arg);static curl_context_t* create_curl_context(curl_socket_t sockfd)
{curl_context_t *context;context = (curl_context_t *) malloc(sizeof(*context));context->sockfd = sockfd;context->event = event_new(base, sockfd, 0, curl_perform, context);return context;
}static void destroy_curl_context(curl_context_t *context)
{event_del(context->event);event_free(context->event);free(context);
}static void add_download(const char *url, int num)
{char filename[50];FILE *file;CURL *handle;snprintf(filename, 50, "%d.download", num);file = fopen(filename, "wb");if(!file) {fprintf(stderr, "Error opening %s\n", filename);return;}handle = curl_easy_init();curl_easy_setopt(handle, CURLOPT_WRITEDATA, file);curl_easy_setopt(handle, CURLOPT_PRIVATE, file);curl_easy_setopt(handle, CURLOPT_URL, url);curl_multi_add_handle(curl_handle, handle);fprintf(stderr, "Added download %s -> %s\n", url, filename);
}static void check_multi_info(void)
{char *done_url;CURLMsg *message;int pending;CURL *easy_handle;FILE *file;while((message = curl_multi_info_read(curl_handle, &pending))) {switch(message->msg) {case CURLMSG_DONE:/* Do not use message data after calling curl_multi_remove_handle() andcurl_easy_cleanup(). As per curl_multi_info_read() docs:"WARNING: The data the returned pointer points to will not survivecalling curl_multi_cleanup, curl_multi_remove_handle orcurl_easy_cleanup." */easy_handle = message->easy_handle;curl_easy_getinfo(easy_handle, CURLINFO_EFFECTIVE_URL, &done_url);curl_easy_getinfo(easy_handle, CURLINFO_PRIVATE, &file);printf("%s DONE\n", done_url);curl_multi_remove_handle(curl_handle, easy_handle);curl_easy_cleanup(easy_handle);if(file) {fclose(file);}break;default:fprintf(stderr, "CURLMSG default\n");break;}}
}static void curl_perform(int fd, short event, void *arg)
{int running_handles;int flags = 0;curl_context_t *context;if(event & EV_READ)flags |= CURL_CSELECT_IN;if(event & EV_WRITE)flags |= CURL_CSELECT_OUT;context = (curl_context_t *) arg;curl_multi_socket_action(curl_handle, context->sockfd, flags,&running_handles);check_multi_info();
}static void on_timeout(evutil_socket_t fd, short events, void *arg)
{int running_handles;curl_multi_socket_action(curl_handle, CURL_SOCKET_TIMEOUT, 0,&running_handles);check_multi_info();
}static int start_timeout(CURLM *multi, long timeout_ms, void *userp)
{if(timeout_ms < 0) {evtimer_del(timeout);}else {if(timeout_ms == 0)timeout_ms = 1; /* 0 means directly call socket_action, but we'll do itin a bit */struct timeval tv;tv.tv_sec = timeout_ms / 1000;tv.tv_usec = (timeout_ms % 1000) * 1000;evtimer_del(timeout);evtimer_add(timeout, &tv);}return 0;
}static int handle_socket(CURL *easy, curl_socket_t s, int action, void *userp,void *socketp)
{curl_context_t *curl_context;int events = 0;switch(action) {case CURL_POLL_IN:case CURL_POLL_OUT:case CURL_POLL_INOUT:curl_context = socketp ?(curl_context_t *) socketp : create_curl_context(s);curl_multi_assign(curl_handle, s, (void *) curl_context);if(action != CURL_POLL_IN)events |= EV_WRITE;if(action != CURL_POLL_OUT)events |= EV_READ;events |= EV_PERSIST;event_del(curl_context->event);event_assign(curl_context->event, base, curl_context->sockfd, events,curl_perform, curl_context);event_add(curl_context->event, NULL);break;case CURL_POLL_REMOVE:if(socketp) {event_del(((curl_context_t*) socketp)->event);destroy_curl_context((curl_context_t*) socketp);curl_multi_assign(curl_handle, s, NULL);}break;default:abort();}return 0;
}int main(int argc, char **argv)
{if(argc <= 1)return 0;if(curl_global_init(CURL_GLOBAL_ALL)) {fprintf(stderr, "Could not init curl\n");return 1;}base = event_base_new();timeout = evtimer_new(base, on_timeout, NULL);curl_handle = curl_multi_init();curl_multi_setopt(curl_handle, CURLMOPT_SOCKETFUNCTION, handle_socket);curl_multi_setopt(curl_handle, CURLMOPT_TIMERFUNCTION, start_timeout);while(argc-- > 1) {add_download(argv[argc], argc);}event_base_dispatch(base);curl_multi_cleanup(curl_handle);event_free(timeout);event_base_free(base);libevent_global_shutdown();curl_global_cleanup();return 0;
}
multi interface code doing two parallel HTTP transfers
/**************************************************************************** _ _ ____ _* Project ___| | | | _ \| |* / __| | | | |_) | |* | (__| |_| | _ <| |___* \___|\___/|_| \_\_____|** Copyright (C) 1998 - 2018, Daniel Stenberg, <daniel@haxx.se>, et al.** This software is licensed as described in the file COPYING, which* you should have received as part of this distribution. The terms* are also available at https://curl.haxx.se/docs/copyright.html.** You may opt to use, copy, modify, merge, publish, distribute and/or sell* copies of the Software, and permit persons to whom the Software is* furnished to do so, under the terms of the COPYING file.** This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY* KIND, either express or implied.****************************************************************************/
/* <DESC>* multi interface code doing two parallel HTTP transfers* </DESC>*/
#include <stdio.h>
#include <string.h>/* somewhat unix-specific */
#include <sys/time.h>
#include <unistd.h>/* curl stuff */
#include <curl/curl.h>/** Simply download two HTTP files!*/
int main(void)
{CURL *http_handle;CURL *http_handle2;CURLM *multi_handle;int still_running = 0; /* keep number of running handles */http_handle = curl_easy_init();http_handle2 = curl_easy_init();/* set options */curl_easy_setopt(http_handle, CURLOPT_URL, "https://10.170.6.66/");/* set options */curl_easy_setopt(http_handle2, CURLOPT_URL, "http://localhost/");/* init a multi stack */multi_handle = curl_multi_init();/* add the individual transfers */curl_multi_add_handle(multi_handle, http_handle);curl_multi_add_handle(multi_handle, http_handle2);/* we start some action by calling perform right away */curl_multi_perform(multi_handle, &still_running);while(still_running) {struct timeval timeout;int rc; /* select() return code */CURLMcode mc; /* curl_multi_fdset() return code */fd_set fdread;fd_set fdwrite;fd_set fdexcep;int maxfd = -1;long curl_timeo = -1;FD_ZERO(&fdread);FD_ZERO(&fdwrite);FD_ZERO(&fdexcep);/* set a suitable timeout to play around with */timeout.tv_sec = 1;timeout.tv_usec = 0;curl_multi_timeout(multi_handle, &curl_timeo);if(curl_timeo >= 0) {timeout.tv_sec = curl_timeo / 1000;if(timeout.tv_sec > 1)timeout.tv_sec = 1;elsetimeout.tv_usec = (curl_timeo % 1000) * 1000;}/* get file descriptors from the transfers */mc = curl_multi_fdset(multi_handle, &fdread, &fdwrite, &fdexcep, &maxfd);if(mc != CURLM_OK) {fprintf(stderr, "curl_multi_fdset() failed, code %d.\n", mc);break;}/* On success the value of maxfd is guaranteed to be >= -1. We callselect(maxfd + 1, ...); specially in case of (maxfd == -1) there areno fds ready yet so we call select(0, ...) --or Sleep() on Windows--to sleep 100ms, which is the minimum suggested value in thecurl_multi_fdset() doc. */if(maxfd == -1) {
#ifdef _WIN32Sleep(100);rc = 0;
#else/* Portable sleep for platforms other than Windows. */struct timeval wait = { 0, 100 * 1000 }; /* 100ms */rc = select(0, NULL, NULL, NULL, &wait);
#endif}else {/* Note that on some platforms 'timeout' may be modified by select().If you need access to the original value save a copy beforehand. */rc = select(maxfd + 1, &fdread, &fdwrite, &fdexcep, &timeout);}switch(rc) {case -1:/* select error */break;case 0:default:/* timeout or readable/writable sockets */curl_multi_perform(multi_handle, &still_running);break;}}curl_multi_cleanup(multi_handle);curl_easy_cleanup(http_handle);curl_easy_cleanup(http_handle2);return 0;
}
multi interface and debug callback
/**************************************************************************** _ _ ____ _* Project ___| | | | _ \| |* / __| | | | |_) | |* | (__| |_| | _ <| |___* \___|\___/|_| \_\_____|** Copyright (C) 1998 - 2018, Daniel Stenberg, <daniel@haxx.se>, et al.** This software is licensed as described in the file COPYING, which* you should have received as part of this distribution. The terms* are also available at https://curl.haxx.se/docs/copyright.html.** You may opt to use, copy, modify, merge, publish, distribute and/or sell* copies of the Software, and permit persons to whom the Software is* furnished to do so, under the terms of the COPYING file.** This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY* KIND, either express or implied.****************************************************************************/
/* <DESC>* multi interface and debug callback* </DESC>*/#include <stdio.h>
#include <string.h>/* somewhat unix-specific */
#include <sys/time.h>
#include <unistd.h>/* curl stuff */
#include <curl/curl.h>typedef char bool;
#define TRUE 1static
void dump(const char *text,FILE *stream, unsigned char *ptr, size_t size,bool nohex)
{size_t i;size_t c;unsigned int width = 0x10;if(nohex)/* without the hex output, we can fit more on screen */width = 0x40;fprintf(stream, "%s, %10.10lu bytes (0x%8.8lx)\n",text, (unsigned long)size, (unsigned long)size);for(i = 0; i<size; i += width) {fprintf(stream, "%4.4lx: ", (unsigned long)i);if(!nohex) {/* hex not disabled, show it */for(c = 0; c < width; c++)if(i + c < size)fprintf(stream, "%02x ", ptr[i + c]);elsefputs(" ", stream);}for(c = 0; (c < width) && (i + c < size); c++) {/* check for 0D0A; if found, skip past and start a new line of output */if(nohex && (i + c + 1 < size) && ptr[i + c] == 0x0D &&ptr[i + c + 1] == 0x0A) {i += (c + 2 - width);break;}fprintf(stream, "%c",(ptr[i + c] >= 0x20) && (ptr[i + c]<0x80)?ptr[i + c]:'.');/* check again for 0D0A, to avoid an extra \n if it's at width */if(nohex && (i + c + 2 < size) && ptr[i + c + 1] == 0x0D &&ptr[i + c + 2] == 0x0A) {i += (c + 3 - width);break;}}fputc('\n', stream); /* newline */}fflush(stream);
}static
int my_trace(CURL *handle, curl_infotype type,unsigned char *data, size_t size,void *userp)/* debug回调函数 */
{const char *text;(void)userp;(void)handle; /* prevent compiler warning */switch(type) {case CURLINFO_TEXT:fprintf(stderr, "== Info: %s", data);/* FALLTHROUGH */default: /* in case a new one is introduced to shock us */return 0;case CURLINFO_HEADER_OUT:text = "=> Send header";break;case CURLINFO_DATA_OUT:text = "=> Send data";break;case CURLINFO_HEADER_IN:text = "<= Recv header";break;case CURLINFO_DATA_IN:text = "<= Recv data";break;}dump(text, stderr, data, size, TRUE);return 0;
}/** Simply download a HTTP file.*/
int main(void)
{CURL *http_handle;CURLM *multi_handle;int still_running = 0; /* keep number of running handles */http_handle = curl_easy_init();/* set the options (I left out a few, you'll get the point anyway) */curl_easy_setopt(http_handle, CURLOPT_URL, "https://10.170.6.66/");curl_easy_setopt(http_handle, CURLOPT_DEBUGFUNCTION, my_trace);/* debug回调函数 */curl_easy_setopt(http_handle, CURLOPT_VERBOSE, 1L);//CURLOPT_VERBOSE:详细/* init a multi stack */multi_handle = curl_multi_init();/* add the individual transfers */curl_multi_add_handle(multi_handle, http_handle);/* 添加一个handle *//* we start some action by calling perform right away */curl_multi_perform(multi_handle, &still_running);/* 执行 */while(still_running) {struct timeval timeout;int rc; /* select() return code */CURLMcode mc; /* curl_multi_fdset() return code */fd_set fdread;fd_set fdwrite;fd_set fdexcep;int maxfd = -1;long curl_timeo = -1;FD_ZERO(&fdread);FD_ZERO(&fdwrite);FD_ZERO(&fdexcep);/* set a suitable timeout to play around with */timeout.tv_sec = 1;timeout.tv_usec = 0;curl_multi_timeout(multi_handle, &curl_timeo);if(curl_timeo >= 0) {timeout.tv_sec = curl_timeo / 1000;if(timeout.tv_sec > 1)timeout.tv_sec = 1;elsetimeout.tv_usec = (curl_timeo % 1000) * 1000;}/* get file descriptors from the transfers */mc = curl_multi_fdset(multi_handle, &fdread, &fdwrite, &fdexcep, &maxfd);if(mc != CURLM_OK) {fprintf(stderr, "curl_multi_fdset() failed, code %d.\n", mc);break;}/* On success the value of maxfd is guaranteed to be >= -1. We callselect(maxfd + 1, ...); specially in case of (maxfd == -1) there areno fds ready yet so we call select(0, ...) --or Sleep() on Windows--to sleep 100ms, which is the minimum suggested value in thecurl_multi_fdset() doc. */if(maxfd == -1) {
#ifdef _WIN32Sleep(100);rc = 0;
#else/* Portable sleep for platforms other than Windows. */struct timeval wait = { 0, 100 * 1000 }; /* 100ms */rc = select(0, NULL, NULL, NULL, &wait);
#endif}else {/* Note that on some platforms 'timeout' may be modified by select().If you need access to the original value save a copy beforehand. */rc = select(maxfd + 1, &fdread, &fdwrite, &fdexcep, &timeout);}switch(rc) {case -1:/* select error */still_running = 0;printf("select() returns error, this is badness\n");break;case 0:default:/* timeout or readable/writable sockets */curl_multi_perform(multi_handle, &still_running);/* 执行 */break;}}curl_multi_cleanup(multi_handle);curl_easy_cleanup(http_handle);return 0;
}
A basic application source code using the multi interface doing two transfers in parallel.
/**************************************************************************** _ _ ____ _* Project ___| | | | _ \| |* / __| | | | |_) | |* | (__| |_| | _ <| |___* \___|\___/|_| \_\_____|** Copyright (C) 1998 - 2019, Daniel Stenberg, <daniel@haxx.se>, et al.** This software is licensed as described in the file COPYING, which* you should have received as part of this distribution. The terms* are also available at https://curl.haxx.se/docs/copyright.html.** You may opt to use, copy, modify, merge, publish, distribute and/or sell* copies of the Software, and permit persons to whom the Software is* furnished to do so, under the terms of the COPYING file.** This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY* KIND, either express or implied.****************************************************************************/
/* <DESC>* A basic application source code using the multi interface doing two* transfers in parallel.* </DESC>*/#include <stdio.h>
#include <string.h>/* somewhat unix-specific */
#include <sys/time.h>
#include <unistd.h>/* curl stuff */
#include <curl/curl.h>/** Download a HTTP file and upload an FTP file simultaneously.*/#define HANDLECOUNT 2 /* Number of simultaneous transfers */
#define HTTP_HANDLE 0 /* Index for the HTTP transfer */
#define FTP_HANDLE 1 /* Index for the FTP transfer */int main(void)
{CURL *handles[HANDLECOUNT];CURLM *multi_handle; /* 多任务句柄 */int still_running = 0; /* keep number of running handles */int i;CURLMsg *msg; /* for picking up messages with the transfer status */int msgs_left; /* how many messages are left *//* Allocate one CURL handle per transfer */for(i = 0; i<HANDLECOUNT; i++)handles[i] = curl_easy_init();/* 单任务句柄 *//* set the options (I left out a few, you'll get the point anyway) */curl_easy_setopt(handles[HTTP_HANDLE], CURLOPT_URL, "https://10.170.6.66/");/* http任务 */curl_easy_setopt(handles[FTP_HANDLE], CURLOPT_URL, "https://10.170.6.66/");/* ftp任务，先用http */curl_easy_setopt(handles[FTP_HANDLE], CURLOPT_UPLOAD, 1L); /* ftp上传arg *//* init a multi stack */multi_handle = curl_multi_init();/* 多任务句柄 *//* add the individual transfers */for(i = 0; i<HANDLECOUNT; i++)curl_multi_add_handle(multi_handle, handles[i]);/* 将任务添加至多任务句柄中 *//* we start some action by calling perform right away */curl_multi_perform(multi_handle, &still_running);/* 多任务执行 */while(still_running) {/* */struct timeval timeout;int rc; /* select() return code */CURLMcode mc; /* curl_multi_fdset() return code */fd_set fdread;fd_set fdwrite;fd_set fdexcep;int maxfd = -1;long curl_timeo = -1;FD_ZERO(&fdread);FD_ZERO(&fdwrite);FD_ZERO(&fdexcep);/* set a suitable timeout to play around with */timeout.tv_sec = 1;timeout.tv_usec = 0;curl_multi_timeout(multi_handle, &curl_timeo);/* 设置超时 */if(curl_timeo >= 0) {timeout.tv_sec = curl_timeo / 1000;if(timeout.tv_sec > 1)timeout.tv_sec = 1;elsetimeout.tv_usec = (curl_timeo % 1000) * 1000;}/* get file descriptors from the transfers */mc = curl_multi_fdset(multi_handle, &fdread, &fdwrite, &fdexcep, &maxfd);/* 多任务select fdset 设置 */if(mc != CURLM_OK) {fprintf(stderr, "curl_multi_fdset() failed, code %d.\n", mc);break;}/* On success the value of maxfd is guaranteed to be >= -1. We callselect(maxfd + 1, ...); specially in case of (maxfd == -1) there areno fds ready yet so we call select(0, ...) --or Sleep() on Windows--to sleep 100ms, which is the minimum suggested value in thecurl_multi_fdset() doc. */if(maxfd == -1) {
#ifdef _WIN32Sleep(100);rc = 0;
#else/* Portable sleep for platforms other than Windows. */struct timeval wait = { 0, 100 * 1000 }; /* 100ms */rc = select(0, NULL, NULL, NULL, &wait);/* 等待时间：可以用sleep */
#endif}else {/* Note that on some platforms 'timeout' may be modified by select().If you need access to the original value save a copy beforehand. */rc = select(maxfd + 1, &fdread, &fdwrite, &fdexcep, &timeout);/* select多路复用 */}switch(rc) {case -1:/* select error */break;case 0: /* timeout */default: /* action */curl_multi_perform(multi_handle, &still_running);break;}}/* See how the transfers went */while((msg = curl_multi_info_read(multi_handle, &msgs_left))) {if(msg->msg == CURLMSG_DONE) {int idx;/* Find out which handle this message is about */for(idx = 0; idx<HANDLECOUNT; idx++) {int found = (msg->easy_handle == handles[idx]);if(found)break;}switch(idx) {case HTTP_HANDLE:printf("HTTP transfer completed with status %d\n", msg->data.result);break;case FTP_HANDLE:printf("FTP transfer completed with status %d\n", msg->data.result);break;}}}curl_multi_cleanup(multi_handle);/* Free the CURL handles */for(i = 0; i<HANDLECOUNT; i++)curl_easy_cleanup(handles[i]);return 0;
}
C语言libcurl例程：multi 多线程，多任务相关推荐
- 哲学家就餐问题C语言实现（涉及多线程、信号量等）
哲学家就餐问题C语言实现(涉及多线程.信号量等) 1.实验原理 2.实验内容 3.代码部分 3.1 重要代码展示 3.2 源码 3.3 运行结果展示 1.实验原理 由Dijkstra提出并解决的哲 ...
- c语言程序stm8s,stm8s的c语言编程例程
stm8s的c语言编程例程 实例一:控制灯的亮灭(或者蜂鸣器响,只要连接相应端口就可以了): #i nclude "stm8s.h""头文件 #defi ne ulong ...
- Windows多线程多任务设计初步zz
［前言:］当前流行的Windows操作系统,它能同时运行几个程序(独立运行的程序又称之为进程),对于同一个程序,它又可以分成若干个独立的执行流,我们称之为线程,线程提供了多任务处理的能力.用进程和线程 ...
- sonix c语言 pdf,松翰c语言(项目)例程（Sonix C language routines (project)）.doc
松翰c语言(项目)例程(Sonix C language routines (project)) 松翰c语言(项目)例程(Sonix C language routines (project)) So ...
- 单片机c语言孔雀开屏,单片机C语言案例程教学指南.doc
单片机C语言案例程教学指南 单片机C语言案例教程教学指南 一 本书的特色和教学目标 本书是中职学校电子专业的单片机基础教材.特色在于全书以"案例+项目"的形式展开,由26个案例和5 ...
- c语言文件发送程序,C语言程序例程的文件结构
以往写C语言程序都是一个文件里面写个几十.几百行,在练算法的时候还可以,现在搞开发需要模块化编程,所谓模块化编程,就是指一个程序包含多个源文件(.c 文件和 .h 文件),每个 .c 文件可以被称为一 ...
- 向你的C语言项目中加入多线程
整理自网络 作者: 赵岩/Serdar等 01.为什么C语言不会过时 评价任何一门编程语言,都是招人骂的.永远是这样.就像是春寒料峭的季节,街上穿棉袄和穿单衣的擦肩而过,双方一定是同时在心里出现了两 ...
- 超详细c语言简化tcp通信接口(多线程实现一个服务端处理多个客户端服务)
超详细c语言tcp通信接口 1.可下载源码(客户端 || 服务端通信) 2.说明 3.接口代码 4.客户端通信main_client_demo.c 5.服务端通信main_server_demo.c ...
- 例程五、HTTP Range使用 分段下载 与 下载后c语言合成例程 以照片为例
1. HTTP分段下载 可靠reference: https://www.bbsmax.com/A/gVdnERW15W/ https://developer.mozilla.org/zh-CN/do ...
最新文章
- sqlite3 内存持续增加_sqlite3使用简介(内含解决sqlite内存的方法)
- 从PM到GAN——LSTM之父Schmidhuber横跨22年的怨念（文字版）
- 基础编程题目集 7-3 逆序的三位数 (10 分)
- Geatpy自定义初始种群
- BZOJ3938 UOJ88：[集训队互测2015]Robot——题解
- Java、JSP水费管理系统
- 灰度发布 java_灰度发布系统的实现步骤
- 1.8 微信小程序 使用canvas绘制奥运会五环
- 小米5-root-记录
- 服务器的丢包率一般与什么有关
- JavaScript+css实现的喜庆活动邀请函多页面html源码
- 在控制台打印不完整的解决问题
- 说说JS在IE中的内存泄露问题
- ORACLE日期函数浅谈
- SOA详解微服务与SOA的关系
- KingbaseES共享集群存储方案的选择
- Win7 64位下Kindle4、闪迪Cruzer BladeU盘等无法安装驱动问题的解决方法
- 如何读一篇优秀的计算机论文？
- svg绘制小宝宝爱心动画
- 记录安卓开发的好网站