平面包装设计怎么制作

As developers of SQL Server Integrations Services (SSIS) solutions, we have more than likely configured Flat File Connections as sources in our Data Flow Tasks. Whilst any unforeseen changes to the structure and formatting of flat files will invariably cause SSIS packages to fail, as developers we can still do a lot in reducing unnecessary SSIS package failures relating to data coming out of flat files. In this article, we offer recommended development practices to some flat file source errors that occur as a result of poor SSIS development practices.

作为SQL Server Integrations Services （SSIS）解决方案的开发人员，我们很可能已将Flat File Connections配置为Data Flow Task中的源 。 尽管对平面文件的结构和格式进行任何不可预见的更改都将不可避免地导致SSIS包失败，但作为开发人员，我们仍然可以在减少与平面文件中的数据相关的不必要的SSIS包失败方面做很多事情。 在本文中，我们为由于不良SSIS开发实践而发生的一些平面文件源错误提供了推荐的开发实践。

平面文件源错误1：系统找不到指定的文件 (Flat File Source Error #1: The system cannot find the file specified)

This typically happens when a hardcoded flat file connection specified at design time is not found by SSIS during package execution. For instance, in the package shown in Figure 1, I have configured my Data Flow Task to extract a list of nominated players for the 2016 FIFA Ballon D’Or.

当在包执行期间SSIS未找到设计时指定的硬编码平面文件连接时，通常会发生这种情况。 例如，在图1所示的程序包中，我配置了数据流任务以提取2016 FIFA Ballon D'Or的提名球员名单。

Prior to executing the package shown in Figure 1, I momentarily relocated the flat file specified in the FIFABallonDOr connection to another location. This causeed the package to fail when I later execute it as shown in Figure 2.

在执行图1所示的包之前，我暂时将FIFABallonDOr连接中指定的平面文件重定位到另一个位置。 这导致程序包在以后执行时失败， 如图2所示。

A summary of the error messages returned in Figure 2 is shown in Figure 3.

图3中显示了图 2中返回的错误消息的摘要。

推荐的实践1：使用ForEach循环容器迭代平面文件 (Recommended Practice #1: Iterate Flat Files Using ForEach Loop Container)

Although for the purposes of simulating this type of an error, I deliberately moved the flat file into another location, the fact is there are plenty of reasons for a flat file not to be found in a specified path (i.e. specified drive becomes corrupt, account access to flat file is revoked etc.). Thus, as developers we must prepare for such errors and one best way to avoid such an error is to use a ForEach Loop Container. The ForEach Loop Container can iterate through flat files and execute the Data Flow Task only when files are found. Figure 4 shows our updated test package using the ForEach Loop Container – as it can be seen, the Data Flow Task is not executed as the flat file was not found.

尽管出于模拟此类错误的目的，我故意将平面文件移动到另一个位置，但事实是有很多原因导致在指定路径中找不到平面文件（即，指定的驱动器已损坏，帐户撤消了对平面文件的访问等）。 因此，作为开发人员，我们必须为此类错误做准备，而避免此类错误的最佳方法是使用ForEach循环容器 。 仅当找到文件时， ForEach循环容器才能遍历平面文件并执行数据流任务 。 图4显示了使用ForEach循环容器更新的测试包–可以看出，由于未找到平面文件，因此未执行数据流任务 。

平面文件源错误2：数据转换失败 (Flat File Source Error #2: Data conversion failed)

There are two areas of the SSIS package that can be affected by the Data conversion failed error. The first part is between the actual raw text file and the flat file connection manager. During the setup of my sample flat file connection manager, the two fields coming from the FIFABallonDOr file were assigned a default length of 50, as shown in Figure 5.

SSIS包的两个区域可能会受到数据转换失败错误的影响。 第一部分在实际的原始文本文件和平面文件连接管理器之间。 在我的样本平面文件连接管理器的设置过程中，来自FIFABallonDOr文件的两个字段被分配了默认长度50， 如图5所示。

However, having later edited the FIFABallonDOr file shown in Figure 5 to include another fictitious nominee in line 5, the package execution fails at Flat File Source component shown in Figure 6.

然而，具有编辑后在图5中示出为包括在第5行另一个虚拟提名的FIFABallonDOr文件，包执行在图6中所示平面文件源组件出现故障。

The reason for this failure is due to possible data truncation errors within the Nominee column as shown in Figure 7.

失败的原因是由于Nominee列中可能存在数据截断错误， 如图7所示。

The second cause of the Data conversion failed error is that even though you may have adjusted the length of the Nominee column to accommodate my fictitious entry, if you don’t refresh the metadata of the Data Flow Task’s Flat File Source component the package will fail because of the synchronisation warning (outdated column metadata) shown in Figure 8.

数据转换失败错误的第二个原因是，即使您已经调整了“ 提名”列的长度以适应我的虚构条目，但如果不刷新数据流任务的“ 平面文件源”组件的元数据，该包也会失败由于出现了图8中所示的同步警告（列元数据已过期）。

建议的练习2：重定向截断行 (Recommended Practice #2: Redirect Truncation Rows)

There is no worst or best practice when dealing with the warning shown in Figure 8, you simply have to refresh the metadata of the Flat File Source component and when that happens, you will receive a popup dialog to confirm the metadata refresh as shown in Figure 9.

在图8所示的警告处理时，没有最坏或最好的做法，你只需要刷新平面文件源组件的元数据，这时，您将收到一个弹出式对话框，确认元数据刷新， 如图9 。

Furthermore, there is no perfect solution to handling the error shown in Figure 7 – one possible solution is to configure an error output by adding another Destination component that will store redirected error rows from source as shown in Figure 10.

此外，没有完美的解决方案来处理图7所示的错误–一种可能的解决方案是通过添加另一个Destination组件来配置错误输出，该组件将存储来自源的重定向错误行， 如图10所示。

You would then have to configure the Error Output setting in Flat File Source Editor to Redirect row as shown in Figure 11.

然后，您必须在Flat File Source Editor中将Error Output设置配置为Redirect行 ， 如图11所示。

After executing the package again, the fictitious row data should be redirected to the error table as shown in Figure 12.

再次执行该程序包后，应将虚拟行数据重定向到错误表， 如图12所示。

You can double-check this by querying the error table which should return the offending record as shown in Figure 13.

您可以通过查询错误表来再次检查该错误表，错误表应返回违规记录， 如图13所示。

The only problem with this workaround is that there could still be a possibility that the offending record is also longer than the length of the corresponding field in the error table, in which case the redirecting of error rows would still fail due to Data conversion failed error.

此解决方法的唯一问题是，有问题的记录仍然可能比错误表中相应字段的长度还要长，在这种情况下，由于数据转换失败 ，错误行的重定向仍将失败。

平面文件源错误3：该进程无法访问该文件，因为该文件正在被另一个进程使用 (Flat File Source Error #3: The process cannot access the file because it is being used by another process)

Another common error relating to working with flat files in SSIS is that you could be processing a file that is still being used by another process. This typically happens when you attempt to process a flat file that is either open or currently being written into by another process. When such a scenario occurs, your SSIS package will throw an exception similar to the message shown in Figure 14.

与在SSIS中使用平面文件有关的另一个常见错误是，您可能正在处理仍由另一个进程使用的文件。 当您尝试处理一个已打开或当前正在由另一个进程写入的平面文件时，通常会发生这种情况。 发生这种情况时，您的SSIS包将引发类似于图14所示消息的异常。

推荐的实践3：使用脚本任务检查平面文件锁定 (Recommended Practice #3: Use Script Task to Check for Flat File Lock)

When data is being read out of a flat file, all that SSIS is doing is merely calling the StreamReader Class from System.IO namespace. Thus, we need to find a mechanism in SSIS Package that will make use of the StreamReader Class to ensure that you are not processing files that are at the time locked by another process. One such mechanism is the Script Task wherein you can set package variables based on the output of the StreamReader Class. To demonstrate such a mechanism, I have updated our test package to include a Script Task step shown in Figure 15.

从平面文件中读取数据时，SSIS所做的只是从System.IO名称空间调用StreamReader类 。 因此，我们需要在SSIS包中找到一种机制，该机制将利用StreamReader类来确保您不会处理当前被另一个进程锁定的文件。 一种这样的机制是脚本任务 ，您可以在其中基于StreamReader类的输出设置程序包变量。 为了演示这种机制，我更新了测试包以包括图15所示的“ 脚本任务”步骤。

The definition of the Script Task’s Main() method is shown Script 1. The script basically set package variable islocked to either true (indicating that flat file is available for use) or false.

脚本1展示了脚本任务的Main（）方法的定义。 脚本基本上将包变量设置为true （表示可以使用平面文件）或false 。

public void Main() {try {string line;System.IO.StreamReader file = new System.IO.StreamReader(@ "C:\temp\New folder\FIFABallonDOr.txt");while ((line = file.ReadLine()) != null) {Dts.Variables["User::islocked"].Value = true;break;}file.Close();} catch (Exception ex) {if (ex.InnerException.Message.Contains("because it is being used by another process")) {Dts.Variables["User::islocked"].Value = false;}}Dts.TaskResult = (int) ScriptResults.Success;
}

Package variable islocked is then used to evaluate the Expression of the Precedence Constraint that joins the Script Task and the Data Flow Task as shown in Figure 16.

然后，使用包变量islocked评估连接脚本任务和数据流任务的优先约束的表达式 ， 如图16所示。

结论 (Conclusion)

The flat file is one of the commonly configured sources of data. However, you may find yourself frequently having to troubleshoot a failing package as a result of poorly designed SSIS package. Fortunately, this article has provided solutions that could be used eliminate some of the errors relating to flat files.

平面文件是通常配置的数据源之一。 但是，由于SSIS软件包的设计不当，您可能会经常不得不对出现故障的软件包进行故障排除。 幸运的是，本文提供了可用于消除与平面文件有关的一些错误的解决方案。

资料下载 (Downloads)

• SQLShack SSIS Sample Solution SQLShack SSIS示例解决方案
• FIFABallonDOr Sample Text FileFIFABallonDOr示例文本文件

参考资料 (References)

• Flat File Source 平面文件源
• StreamReader Class StreamReader类
• Precedence Constraints 优先约束

翻译自: https://www.sqlshack.com/how-to-avoid-package-design-flaws-when-sourcing-data-from-flat-files/

平面包装设计怎么制作