文件传输请求最快的方式_实施块请求并更快地上传大文件30

文件传输请求最快的方式

Our test shows that the implementation of the chunked requests makes the upload 30% faster.

我们的测试表明，分块请求的实现使上载速度提高了30％。

介绍 (Intro)

I am a front-end software engineer at TransPerfect. TransPerfect is language service / localization company. At TransPerfect, I am mainly working on a file sharing platform called GlobalLink Share (GLShare). GLShare has some outstanding features such as anonymous file sharing, guest file sharing, and unlimited file size sharing. Clients use GLShare to share files

我是TransPerfect的前端软件工程师。 TransPerfect是语言服务/本地化公司。在TransPerfect，我主要在一个名为GlobalLink Share(GLShare)的文件共享平台上工作。 GLShare具有一些出色的功能，例如匿名文件共享，来宾文件共享和无限的文件大小共享。客户端使用GLShare共享文件

Recently, I implemented chunked request in GLShare and improves the uploading speed more than 30%. In this article, I want to share my long journey to implement chunked request and what I learned from it.

最近，我在GLShare中实现了分块请求，并将上传速度提高了30％以上。在本文中，我想分享我实现分段请求的漫长旅程以及从中获得的经验。

Note: It’s developed using Vue.js and the code snippets will be written in Vue.js as well

注意：它是使用Vue.js开发的，代码段也将用Vue.js编写

什么是分块请求(What is Chunked Request)

I believe that many of readers are asking “what is Chunked Request?” I had the same question and did not have any of experiences implementing it at all. According to Wikipedia,

我相信许多读者都在问“什么是分块请求？” 我有同样的问题，完全没有经验。根据维基百科，

In chunked transfer encoding, the data stream is divided into a series of non-overlapping “chunks”. The chunks are sent out and received independently of one another

在分块传输编码中，数据流分为一系列不重叠的“块”。 块彼此独立地发送和接收

Chunked transfer encoding is like slicing a baguette and throwing out each slices of the baguette one at a time instead of throwing the whole piece

块状传输编码就像将法棍切成薄片，然后一次扔出法棍的每一片，而不是一整块地扔掉

Scalability would not be the issue for throwing the whole piece of baguette in the real life because the baguette weighs only around few grams. But what if the baguette weights 100 pounds? It will be little bit harder to throw it as a whole chunk

可伸缩性并不是现实生活中将整个法式面包扔掉的问题，因为法式面包仅重约几克。但是，如果法式长棍面包重100磅，该怎么办？将其整体丢掉会有点困难

But what if, the baguette weighs 100 pounds?

但是，如果法式长棍面包重100磅，该怎么办？

The similar issue happens for uploading large ( > 1GB) file. Larger file size gives more stress for both FrontEnd and Backend to upload it. Memory and Timeout issues are the two most prominent issues that can be caused while uploading large file.

上载较大(> 1GB)文件时也会发生类似问题。较大的文件大小会给前端和后端带来更大的上传压力。内存和超时问题是上载大文件时可能引起的两个最突出的问题。

Memory issue for uploading large file

上载大文件的内存问题

Apache and other web servers might start complaining due to the memory usage required if the file size exceeds above 1GB

如果文件大小超过1GB，则由于所需的内存使用量，Apache和其他Web服务器可能会开始抱怨

2. Timeout issue for uploading large file

2.上传大文件超时问题

Uploading a file can take a lot of time. For example, given that a user might have bandwidth in the 100-KBs range, uploading a file around 100MB will take a significant amount of time. Many web servers time out requests after a certain amount of time, which can be as low as 30 seconds, causing the upload to fail

上载文件可能需要很多时间。例如，假设用户的带宽可能在100 KB范围内，则上载大约100MB的文件将花费大量时间。许多Web服务器在一定时间后(可能低至30秒)使请求超时，从而导致上传失败

Therefore, it’s more convenient to slice the file and sends the multiple chunked data one at a time

因此，切片文件并一次发送多个分块数据更为方便。

jQuery插件 (jQuery plugin)

There are some good jQuery plugins available for chunked upload. For example, blueimp’s jQuery-File-The-Upload Git repository has many advanced features for uploading file. It provides chunk uploading with maxChunkSizeproperty and I was almost tempted to integrate blueimp’s jQuery plugin into Vue.js codebase

有一些不错的jQuery插件可用于分块上传。例如，blueimp的jQuery-File-The-Upload Git存储库具有许多用于上传文件的高级功能。它提供了具有maxChunkSize属性的块上传maxChunkSize ，我几乎想将blueimp的jQuery插件集成到Vue.js代码库中

blueimp’s jQuery-File-The-Upload snippet and maxChunkSize property

blueimp的jQuery-File-The-Upload片段和maxChunkSize属性

$('#fileupload').fileupload({    url: 'https://fs-qa.transperfect.com/fs/car-test',    method: 'POST',    maxChunkSize: 100000000 //suggested chunk size 100mb  });$('#fileupload').bind('fileuploadsubmit', function (e, data) {        data.headers = $.extend(data.headers,        {"car-meta-chunk-file-id": generateFileUniqueIdentifier(data),        "Authorization": "Bearer " + token}        );

I actually tried to integrate its jQuery plugin into Vue.js but I was failed completely