介绍 (Introduction)

A few months back, I encountered an interesting challenge at a client site. For those of you whom have read my previous article entitled “Excel in loading multiple workbooks into SQL Server“, you will know that the challenge centered around loading the data from multiple spreadsheets into our SQLShack financial database.

几个月前，我在一个客户站点遇到了一个有趣的挑战。 对于那些阅读了我以前的文章“ 将多个工作簿加载到SQL Server中的Excel ”的人，您将知道挑战集中在将来自多个电子表格的数据加载到我们SQLShack财务数据库中。

Now, one of the enterprises business rules was that the loading of this data was NOT to occur before that last of the daily spreadsheets arrived in the common data repository.

现在，企业业务规则之一是，在最后的每日电子表格到达公用数据存储库之前， 不得进行此数据的加载。

Imagine in your own mind the client had offices in California, Boston, London, Munich, Sydney and Hong Kong. Being a 24 x 7 global organization it is not surprising that the data files will arrive at different time. We are now going to have a look at one innovative way of achieving this load task utilizing Visual Studio as our front end.

您自己想象一下，客户在加利福尼亚，波士顿，伦敦，慕尼黑，悉尼和香港设有办事处。 作为一家24 x 7的全球组织，数据文件将在不同的时间到达也就不足为奇了。 现在，我们将探讨一种利用Visual Studio作为前端来完成此加载任务的创新方法。

The Visual Studio project that we are going to construct, once completed and activated, WILL REMAIN RUNNING continually. It is never stopped. The overhead of this continuous run is minimal as most of the time it remains dormant and is only awakened on the arrival of new files. The arrival of a dummy file called “finish.txt” is the signal to start the actual data processing.

一旦完成并激活，我们将要构建的Visual Studio项目将继续保持运行状态。 它永远不会停止。 连续运行的开销很小，因为在大多数情况下它保持Hibernate状态，并且仅在新文件到达时才被唤醒。 称为“ finish.txt”的虚拟文件的到来是开始实际数据处理的信号。

入门 (Getting Started)

Opening Visual Studio, I am going to create a new VB.net project entitled SQL Shack Global Load.

打开Visual Studio，我将创建一个名为SQL Shack Global Load的新VB.net项目。

My drawing surface is now in view.

现在可以看到我的绘图表面。

It is not necessary to change the size of the “Form” as we shall not be adding any visible controls nor visual objects to the Form.

不必更改“表单”的大小，因为我们不会在表单中添加任何可见控件或视觉对象。

By “Double Clicking” the “FileSystemWatcher” control, the FORM code page is opened (see below).

通过“双击”“ FileSystemWatcher”控件，将打开FORM代码页（请参见下文）。

We now add the System.IO namespace and “Inherits System.Windows.Forms.Form” immediately below the Public Class declaration (see below).

现在，我们在Public Class声明的下面添加System.IO命名空间和“ Inherits System.Windows.Forms.Form” （请参见下文）。

We next declared two local variables which will be utilized later in the process

接下来，我们声明了两个局部变量，这些变量将在后续过程中使用

“TargetDir” is used to point to where the new Excel data spreadsheets are to be found. Once again the data within these spreadsheets is loaded into our SQLShackFinancial database on a daily basis.

“ TargetDir”用于指向要在其中找到新的Excel数据电子表格的位置。 这些电子表格中的数据每天再次被加载到我们SQLShackFinancial数据库中。

“MyWatcher” is merely the name that we have given to our instance of a “FileSystemWatcher” object.

“ MyWatcher”仅仅是我们为“ FileSystemWatcher”对象的实例提供的名称。

Our next task will be to create the necessary code which is to be executed during the “Form” load event (see below).

我们的下一个任务是创建必要的代码，该代码将在“ Form”加载事件期间执行（请参见下文）。

Note that we have “hard wired” the source directory where the Excel spreadsheets are to be found by our load process. Whilst not the best of coding techniques, I wanted to describe what each line of code achieves in the easiest possible manner.

请注意，我们已经“硬接线”了源目录，在该目录中，我们的加载过程将在其中找到Excel电子表格。 虽然不是最好的编码技术，但我想描述每一行代码以最简单的方式实现的目的。

“TargetDir = "C:\SQL Shack\MultipleExcelFileLoad\".

Next we instantiate our instance of the “FileSystemWatcher” essentially telling our ‘solution’ to consider all the files within the given directory “TargetDir”

接下来，我们实例化“ FileSystemWatcher”实例，实际上是告诉我们“解决方案”考虑给定目录“ TargetDir”中的所有文件。

'Create our instance of the FileSystemWatcher.
MyWatcher = New FileSystemWatcher(TargetDir, "*.*")

We next set the “NotifyFilter” property (see above). This tells the “FileSystemWatcher” what type of element(s) it will be reporting back upon. In our case it will be the name of the file(s). After all we are looking for a dummy “txt” file to be inserted into the production spreadsheet Windows directory in order to signify that the SSIS load should now be executed as all the necessary data is present.

接下来，我们设置“ NotifyFilter”属性（请参见上文）。 这告诉“ FileSystemWatcher”它将报告的元素类型。 在我们的情况下，它将是文件的名称。 毕竟，我们正在寻找一个要插入到生产电子表格Windows目录中的虚拟“ txt”文件， 以表明由于存在所有必要数据， 现在应该执行SSIS加载 。

We then enable the “FileSystemWatcher” to process events that occur within the directory.

然后，我们启用“ FileSystemWatcher”来处理目录中发生的事件。

MyWatcher.EnableRaisingEvents = True

解析现有文件 (Parsing the existing files)

The last step within our load event is to parse the names of the files that currently reside within our target directory. A call is made to the “ParseExistingFiles” subroutine (see the code immediately below). As we shall remember “TargetDir” was initialized above.

load事件中的最后一步是解析当前位于目标目录中的文件的名称。 调用“ ParseExistingFiles”子例程（请参见下面的代码）。 我们将记住“ TargetDir”已在上面初始化。

‘ParseExistingFiles(TargetDir)’

The ParseExistingFiles subroutine may be seen below:

可以在下面看到ParseExistingFiles子例程：

We first create “Direktory” as an instance of a “DirectoryInfo” object. FileDetails is set to an array instance which will contain the name of all the files within the given directory (see above).

我们首先创建“ Direktory”作为“ DirectoryInfo”对象的实例。 FileDetails设置为一个数组实例，该实例将包含给定目录中所有文件的名称（请参见上文）。

现在为过程中的肉！ (Now for the meat of the process!!)

The code below is used for the files that are CURRENTLY resident within the directory. In our case the spreadsheets and any other files that may reside within the directory.

下面的代码用于目录中当前驻留的文件。 在我们的情况下，电子表格和目录中可能存在的任何其他文件。

We now loop through the file names contained within the array “FileDetails” ONE NAME AT A TIME and pass EACH name, one by one through to an “are you the filename that I need to start the SSIS daily batch load” subroutine. This subroutine is called ”ParseIndividualFile”:

现在，我们一次遍历包含在数组“ FileDetails”中的一个名称，然后逐个传递每个名称​​到“您是我开始SSIS每日批量加载所需的文件名”子例程的每个名称。 该子例程称为“ ParseIndividualFile” ：

For Each entry As FileInfo In FileDetails
'With the current filename in hand we now check the details of the file to see if it is ‘the “finish file” and if so, then we can start the SSIS package to load
'the spreadsheet data. This is why we now call ParseIndividualFile
ParseIndividualFile(entry.FullName)
Next entry

As we shall note above, the “ParseIndividualFile” subroutine has NOW been added to our source code.

正如我们上面将要指出的，“ ParseIndividualFile”子例程现在已添加到我们的源代码中。

The code for this subroutine is seen below:

该子例程的代码如下所示：

'Now that the current file name has arrived within this sub routine, we need to check its ‘name to see if
'it is "finish.txt". If it is, then the following IF END statement is executed ELSE we ‘continue the loop.If file_name.ToString = "C:\SQL Shack\MultipleExcelFileLoad\finish.txt" ThenProcess.Start("C:\SQL Shack\MultipleExcelFileLoad\Go.bat")'  lstFiles.Items.Add(Now.ToString() & " Processed " & file_name)End If

Once the trigger file arrives, its name will be ‘finish.txt’ and it will therefore be permitted to enter the IF / END loop. Whilst in the loop, a system process is started up and we pass as parameter the name of a Windows DOS batch file (Go.bat) which is to be executed. We shall discuss the contents of this batch file in a section below.

触发文件到达后，其名称将为“ finish.txt”，因此将被允许进入IF / END循环。 在循环中，启动了系统进程，我们将要执行的Windows DOS批处理文件（Go.bat）的名称作为参数传递。 我们将在下面的部分中讨论此批处理文件的内容。

NOW!! , there should only be one trigger file. Once the file has been found, we could then code a breakout from the loop, as any further processing would be pointless. This exercise is left to the reader.

现在！！ ，应该只有一个触发文件。 一旦找到文件，我们就可以对循环中的中断进行编码，因为任何进一步的处理都是没有意义的。 本练习留给读者。

当任何新文件到达目录时 (When any NEW file arrive in the directory)

Thus far, we have failed to cater for newly arriving files. In other words the code that we have created (thus far) caters for files that existed within the directory when the VB application starts or prior to the next run. More than likely we wish to start the VB application running and wait for the file(s) to arrive.

到目前为止，我们仍无法满足新到达的文件。 换句话说，我们创建的代码（到目前为止）迎合了VB应用程序启动时或下次运行之前目录中存在的文件。 我们极有可能希望启动VB应用程序运行并等待文件到达。

This said, we now add another subroutine which ‘reacts’ to the “created event” of the “FileSystemWatcher”. After all, when a file is deposited into the directory is has in fact been “created”. This includes new spreadsheets etc.

也就是说，我们现在在“ FileSystemWatcher”的“创建的事件”中添加另一个“React”的子例程。 毕竟，实际上是在“创建”了文件存放在目录中之后。 这包括新的电子表格等。

The code for this subroutine is shown below.

该子例程的代码如下所示。

Once again a call is made to the “ParseIndividualFile” subroutine to “see” if this is the trigger file.

再次调用“ ParseIndividualFile”子例程以“查看”这是否是触发文件。

The contents of the finish.txt file may be any text that you wish to place in the file. The text is a space filler and is irrelevant to any processing (see below):

finish.txt文件的内容可以是您希望放置在文件中的任何文本。 文本为空格，与任何处理均无关（请参见下文）：

The contents of Go.Bat (the Windows DOS batch file) is as follows:

Go.Bat（Windows DOS批处理文件）的内容如下：

As my version of Office 2013 is a 32 bit version, I must use the 32 bit version of dtexec.exe which is located in the SQL Server subdirectories under ‘program files (x86)’ directory. Note that if you attempt this with the 64bit version of dtexec.exe (and your version of Office is a 32 bit version) you will generate a runtime error (see the Addenda below).

由于我的Office 2013版本是32位版本，因此我必须使用32位版本的dtexec.exe，该文件位于SQL Server子目录中的“程序文件（x86）”目录下。 请注意，如果您尝试使用64位版本的dtexec.exe（而您的Office版本是32位版本），则会生成运行时错误（请参见下面的附录）。

After the package has been executed, we copy the spreadsheets out of the production directory to a backup directory for further processing (however this is out of the scope of the present discourse).

执行完程序包后，我们将电子表格从生产目录复制到备份目录以进行进一步处理（但是，这超出了本文的讨论范围）。

让我们试一下 (Let us give it a test drive)

To begin, our production directory appears as follows:

首先，我们的生产目录如下所示：

Let us start our Visual Studio Project.

让我们开始我们的Visual Studio项目。

Now let us drop the “finish.txt” file into the subdirectory

现在让我们将“ finish.txt”文件放到子目录中

Note that because the trigger file “finish.txt” has arrived that “Go.bat” was executed (see above).

请注意，由于触发文件“ finish.txt”已到达，因此已执行“ Go.bat”（请参见上文）。

The SQL Server tables within the SQLShackFinancial database have been populated (see above).

已填充SQLShackFinancial数据库中SQL Server表（请参见上文）。

Meanwhile back in the production directory things have definitely changed.

同时，在生产目录中，情况肯定发生了变化。

Note that the spreadsheets and the finish /trigger file are missing.

请注意，电子表格和完成/ trigger文件丢失。

The spreadsheets have been moved to the ‘Backups’ directory (see below)

电子表格已移至“备份”目录（请参见下文）

..and the finish / trigger file has been deleted.

..并且完成/触发文件已被删除。

This said we have achieved what we started out to accomplish.

这表示我们已经实现了开始要完成的目标。

结论 (Conclusions)

A few weeks back, we saw how the data from multiple Excel spreadsheets could be loaded into our SQL Server database, utilizing SQL Server Data Tools. At the end of the article, I mentioned that we could automate the load. If we knew with certainty that all of the data would be present in the production directory at a given time, we could have run the load easily enough via the SQL Server Agent.

几周前，我们看到了如何利用SQL Server数据工具将来自多个Excel电子表格的数据加载到我们SQL Server数据库中。 在文章的最后，我提到我们可以自动执行加载。 如果我们确定在给定时间所有数据都将出现在生产目录中，则可以通过SQL Server代理足够轻松地运行加载。

In reality in many 24 x 7 enterprises with offices around the world, have the ‘arrival times’ of their data files varied AND most of these firms wish to process ONLY when all of the data has arrived.

实际上，在世界各地设有办事处的许多24 x 7企业中，其数据文件的“到达时间”各不相同，并且这些公司中的大多数希望仅在所有数据到达后才进行处理。

The processing method discussed above can help you achieve your end goals.

上面讨论的处理方法可以帮助您实现最终目标。

As always, should you wish the code for this exercise, please let me know.

与往常一样，如果您希望该练习的代码，请告诉我。

In the interim, happy programming.

在此期间，编程愉快。

附加物 (Addenda)

Running in 32 bits mode requires a change to the call batch file.

以32位模式运行需要更改呼叫批处理文件。

翻译自: https://www.sqlshack.com/automatically-load-data-into-sql-server-using-visual-studio/