ssis lookup

We will explore Lookup Transformation in SSIS in this article for incremental data loading in SQL Server.

我们将在本文中探索SSIS中的查找转换，以在SQL Server中进行增量数据加载。

In my previous articles, SSIS Multicast Transformation overview and SSIS Conditional Split Transformation overview, we explored the Multicast and Conditional Split Transformations in SSIS. To begin with, let me introduce these transformations.

在我之前的文章SSIS组播转换概述和SSIS条件拆分转换概述中 ，我们探讨了SSIS中的组播和条件拆分转换。 首先，让我介绍这些转换。

• SSIS Multicast: It creates logical copies of the source data for multiple destinations SSIS组播：它为多个目标创建源数据的逻辑副本
• SSIS Conditional Split: It splits the source data into multiple destinations as per the specified conditions SSIS条件拆分：根据指定条件将源数据拆分为多个目标

Let’s go ahead and see Lookup Transformation in action.

让我们继续前进，看看运行中的查找转换。

真实的例子 (The Real-world Example )

Before we start understanding Lookup Transformation in SSIS, let’s understand the situation in which it can be useful.

在开始理解SSIS中的查找转换之前，让我们了解一下它可能有用的情况。

Suppose you have a dbo.books table and it contains the following entry. In this table, the BookID column is an identity column with the primary key.

假设您有一个dbo.books表，并且其中包含以下条目。 在此表中，BookID列是带有主键的标识列。

[BookID]	[Book Title]	[Book Price]
1	SQL World	560
2	Oracle overview	780
3	Explore SSIS	458
4	Data Visualization	679
5	SQL Components	353
6	High Availability	781

[BookID]	[书名]	[图书价格]
1个	SQL世界	560
2	Oracle概述	780
3	探索SSIS	458
4	数据可视化	679
5	SQL组件	353
6	高可用性	781

We want to copy the contents of this table to another destination table dbo.bookshistory, and we have the following conditions.

我们想要将该表的内容复制到另一个目标表dbo.bookshistory，并且具有以下条件。

条件1：如果目标中不存在，请插入所有记录 (Condition 1: Insert all records if it does not present in the destination )

If the destination table (dbo.bookshistory) table does not contain the records similar to the source table (dbo.books), the SSIS package should copy the contents.

如果目标表（dbo.bookshistory）表不包含类似于源表（dbo.books）的记录，则SSIS包应复制内容。

If you execute the SSIS package, the destination table should contain the following records.

如果执行SSIS包，则目标表应包含以下记录。

[BookID]	[Book Title]	[Book Price]
1	SQL World	560
2	Oracle overview	780
3	Explore SSIS	458
4	Data Visualization	679
5	SQL Components	353
6	High Availability	781

[BookID]	[书名]	[图书价格]
1个	SQL世界	560
2	Oracle概述	780
3	探索SSIS	458
4	数据可视化	679
5	SQL组件	353
6	高可用性	781

条件2：仅在目标表中插入新记录 (Condition 2: Insert only new records in the destination table)

If there is already a book entry in the destination table and a new entry is made into the source table, the SSIS package should only insert the new record into the destination table. It should not insert duplicate records in the destination table

如果目标表中已经有书籍条目，并且在源表中创建了新条目，则SSIS包应仅将新记录插入目标表中。 它不应在目标表中插入重复的记录

The source table gets a new entry in the dbo.books table. In the following table, we inserted BookID 7.

源表在dbo.books表中获得一个新条目。 在下表中，我们插入了BookID 7。

[BookID]	[Book Title]	[Book Price]
1	SQL World	560
2	Oracle overview	780
3	Explore SSIS	458
4	Data Visualization	679
5	SQL Components	353
6	High Availability	781
7	The role of DBA	872

[BookID]	[书名]	[图书价格]
1个	SQL世界	560
2	Oracle概述	780
3	探索SSIS	458
4	数据可视化	679
5	SQL组件	353
6	高可用性	781
7	DBA的作用	872

The destination table contains the previous 6 rows. Now, once we execute the SSIS package, it should only insert BookID 7 in the destination table.

目标表包含前6行。 现在，一旦我们执行了SSIS包，它应该只在目标表中插入BookID 7。

[BookID]	[Book Title]	[Book Price]
7	The role of DBA	872

[BookID]	[书名]	[图书价格]
7	DBA的作用	872

条件3：更新目标表中的相关记录 (Condition 3: Update relevant records in the destination table)

If there is any change in the existing data in the source table, it should also get updated in the destination table.

如果源表中的现有数据有任何更改，则它也应在目标表中得到更新。

Suppose the price of a book changes. In this condition, we want to update the specific row as well in the destination. SSIS package should not update remaining unchanged rows in the destination. In the following table, the price of BookID reduces to 510.

假设一本书的价格改变了。 在这种情况下，我们也要更新目标中的特定行。 SSIS包不应更新目标中其余未更改的行。 在下表中，BookID的价格降低到510。

[BookID]	[Book Title]	[Book Price]
1	SQL World	510
2	Oracle overview	780
3	Explore SSIS	458
4	Data Visualization	679
5	SQL Components	353
6	High Availability	781
7	The role of DBA	872

[BookID]	[书名]	[图书价格]
1个	SQL世界	510
2	Oracle概述	780
3	探索SSIS	458
4	数据可视化	679
5	SQL组件	353
6	高可用性	781
7	DBA的作用	872

You execute the SSIS package, and it should perform the update only for BookID 1.

您执行SSIS包，并且它应该仅对BookID 1执行更新。

[BookID]	[Book Title]	[Book Price]
1	SQL World	510

[BookID]	[书名]	[图书价格]
1个	SQL世界	510

We can use the T-SQL code for specific conditions, but it is a complicated task. We need to write up code to perform data comparisons and then further insert and update accordingly.

我们可以将T-SQL代码用于特定条件，但这是一项复杂的任务。 我们需要编写代码以执行数据比较，然后进一步进行插入和更新。

Could we use the SSIS package in this case? How do we meet the requirement?

在这种情况下，我们可以使用SSIS包吗？ 我们如何满足要求？

在SSIS中使用查找转换 (Use of Lookup Transformation in SSIS)

The Lookup Transformation in SSIS is a powerful and useful SSIS transformation to compare the source and destination data. It filters out the matched and unmatched data in the specified destinations.

SSIS中的查找转换是一种功能强大且有用的SSIS转换，用于比较源数据和目标数据。 它将筛选出指定目标中匹配和不匹配的数据。

Let’s create the source table and insert data into it with the following queries.

让我们创建源表，并通过以下查询将数据插入其中。

Create table dbo.books(    BookID int primary key clustered,[Book Title] NVARCHAR(100),[Book Price] money)Insert into dbo.books values (1,'SQL World',560),(2,'Oracle overview',780),(3,'Explore SSIS',458),(4,'Data Visualization',679),(5,'SQL Components',353),(6,'High Availability',781)

We create the following destination table as well but do not insert any data using the T-SQL query.

我们也创建以下目标表，但不使用T-SQL查询插入任何数据。

Create table dbo.bookshistory(   BookID int primary key clustered,[Book Title] nvarchar(100),[Book Price] money)

在条件1的SSIS中创建用于查找转换的SSIS包 (Create an SSIS package for Lookup Transformation in SSIS for Condition 1)

Use the following steps to create an SSIS package.

使用以下步骤创建一个SSIS包。

• Open SQL Server Data Tools and create a new integration project 打开SQL Server数据工具并创建一个新的集成项目

• Drag a Data Flow Task from the SSIS toolbox to the Control Flow

  将数据流任务从SSIS工具箱拖动到控制流

  

• Right-click on Data Flow Task and rename it to SSIS LOOKUP TRANSFORMATION

  右键单击“数据流任务”，然后将其重命名为SSIS LOOKUP TRANSFORMATION

  

• Double click on this task, and it moves to the Data Flow tab

  双击此任务，它将移至“数据流”选项卡

  

Drag an OLE DB Source in Data Flow Task and configure it to provide the OLE DB connection manager and name of the table or view, in our case, it is table books.

在“数据流任务”中拖动一个OLE DB源并将其配置为提供OLE DB连接管理器以及表或视图的名称（在我们的示例中是表书）。

After the configuration, rename the OLE DB Source to SourceData as shown in the following image.

配置后，将OLE DB源重命名为SourceData，如下图所示。

Now, add a Lookup task in the control flow and join the SourceData task with this Lookup task using a green arrow.

现在，在控制流中添加一个Lookup任务，并使用绿色箭头将SourceData任务与此Lookup任务一起加入。

To configure the Lookup task, double click on it, and it opens the following Lookup Transformation Editor.

要配置查阅任务，请双击它，然后它会打开以下查阅变换编辑器。

In Lookup Transformation Editor, We specify the connection manager and cache type. These details are outside the scope of this article and for now we will go with the default values of these.

在“ 查找转换编辑器”中，我们指定连接管理器和缓存类型。 这些细节不在本文讨论范围之内，现在我们将使用它们的默认值。

In the drop-down of Specify how to handle rows with no matching entries, select the following option

在指定如何处理没有匹配条目的行的下拉列表中，选择以下选项

• Redirect rows to no match output 将行重定向到无匹配输出

In condition 1 mentioned above, we want the rows from source to destination table only if the destination table does not have it already. This Redirect rows to no match output compare both the tables and if it finds any difference then only it forwards the rows to the destination table.

在上面提到的条件1中，仅当目标表还没有行时，才希望从源表到目标表的行。 此“ 将行重定向到不匹配的输出”会比较两个表，如果发现有任何差异，则只有将行转发到目标表。

Click on Connections and specify the destination table from the drop-down. We have both the source and destination tables in the same database. You can create a separate connection here as well.

单击连接，然后从下拉列表中指定目标表。 我们在同一数据库中具有源表和目标表。 您也可以在此处创建一个单独的连接。

Click on the Columns, and it shows the source and destination table. We want to compare the BookID columns on both tables. Drag on the BookID column from the source and move it to the destination BookID column. It draws an arrow, as shown in the following image.

单击列，它显示源表和目标表。 我们要比较两个表上的BookID列。 从源上拖动BookID列，然后将其移动到目标BookID列。 它将绘制一个箭头，如下图所示。

Click OK. The Red Cross icon on Lookup does not show after the configuration.

单击确定。 配置后，“查找”上的“红十字”图标不会显示。

Add an OLE DB destination from the SSIS Toolbox. Join the Lookup task with the OLE DB destination, and it opens the following Input-Output selection.

从SSIS工具箱中添加OLE DB目标。 将Lookup任务与OLE DB目标一起加入，它将打开以下Input-Output选择。

Select the following values.

选择以下值。

• Output: Lookup No Match Output 输出：查找无匹配输出
• Input: OLE DB Destination Input 输入： OLE DB目标输入

Click OK and configure the destination table in the OLE DB Destination.

单击“确定”，然后在“ OLE DB目标”中配置目标表。

Look at the arrow between Lookup and destination, and it shows Looking No Match Output.

查看“查找”和“目的地”之间的箭头，该箭头显示“ 查找不匹配输出” 。

Click on START to execute the package. In the following screenshot, we can see that it transferred 6 rows from the source to the destination because the destination table was empty before the package execution.

单击开始以执行程序包。 在下面的屏幕截图中，我们可以看到它从源向目标传输了6行，因为在执行包之前目标表为空。

It transferred all unmatched rows (6 rows) to the destination table.

它将所有不匹配的行（6行）传输到目标表。

在条件2的SSIS中创建用于查找转换的SSIS包 (Create an SSIS package for Lookup Transformation in SSIS for Condition 2)

Let’s add another record in the destination table (as specified in condition 2) using Lookup Transformation in SSIS.

让我们使用SSIS中的Lookup Transformation在目标表中添加另一条记录（如条件2中所指定）。

Insert into dbo.books values (7,'The role of DBA',872)

At this point, we have BookID 7 in the source table, but it is not available in the destination table.

此时，我们在源表中有BookID 7，但在目标表中不可用。

Let’s rerun the package and see how it works.

让我们重新运行该程序包，看看它是如何工作的。

In the following screenshot, observe the following.

在以下屏幕截图中，请注意以下几点。

• The source table has seven rows 源表有七行

• Lookup transformation checks both tables and finds a new record in the destination, and it only inserts the new record in the destination table. It satisfies our condition 2

  查找转换检查两个表并在目标中找到新记录，并且仅将新记录插入目标表中。 满足我们的条件2

  

为条件3中的SSIS创建用于查找转换的SSIS包 (Create an SSIS package for Lookup Transformation in SSIS for Condition 3)

We used the SSIS Lookup for conditions 1 and 2. Suppose we update a record in the source table and we want to update that record in the destination table as well.

我们对条件1和2使用了SSIS查找。假设我们更新了源表中的一条记录，并且我们也想更新目标表中的该记录。

Execute the following query to update the price of BookID 1 to 510.

执行以下查询以将BookID 1的价格更新为510。

UPDATE dbo.booksSET [Book Price] = 510WHERE BookID = 1;

To add an update task, drag an OLE DB Command task and join the Lookup transformation in SSIS for Lookup Match Output.

若要添加更新任务，请拖动OLE DB命令任务，并在SSIS中加入Lookup转换以获得Lookup Match Output。

Rename the OLE Command task as Update Records.

重命名OLE命令任务为更新记录。

Use the following query to create a stored procedure that updates the record in the destination table.

使用以下查询创建一个存储过程，该存储过程将更新目标表中的记录。

USE [SQLShackDemo];GOCREATE PROCEDURE Update_bookshistory @BookID    INT, @booktitle NVARCHAR(50), @Bookprice MONEYASUPDATE [dbo].[bookshistory]SET [Book Title] = @booktitle, [Book Price] = @BookpriceWHERE [BookID] = @BookID;GO

Double click on Update Records, and it opens the Advanced Editor.

双击更新记录，它会打开高级编辑器。

In the Connection Manager, select the connection from the drop-down.

在连接管理器中，从下拉列表中选择连接。

In the Component Properties, add the following SQL command

在组件属性中，添加以下SQL命令

Exec Update_bookhistory (?,?,?)

执行Update_bookhistory（？，？，？）

We use the question mark (?) for the parameters.

我们使用问号（？）作为参数。

In the next tab, Column Mappings, map the input and destination column. We require to map the columns as per defined in the stored procedure. For example, my first parameter is @BookID, and it maps with the [BookID].

在下一个标签列映射中，映射输入列和目标列。 我们需要按照存储过程中定义的方式映射列。 例如，我的第一个参数是@BookID，它与[BookID]映射。

Click Ok, and you can see following the SSIS package.

单击“确定”，您将看到以下SSIS包。

Execute the SSIS package, and we see that it updates all the seven records present in the destination table.

执行SSIS包，我们看到它更新了目标表中存在的所有七个记录。

In the following screenshot, we can verify that the update flows successfully from the source to the destination table.

在以下屏幕截图中，我们可以验证更新是否已成功从源流向目标表。

为条件3在SSIS中创建用于查找转换的SSIS包以仅更新相关行 (Create an SSIS package for Lookup Transformation in SSIS for Condition 3 to update only relevant rows)

We updated a single record in the source table, and it should update only that particular record. It updated all the records and that we might not want to update all records, especially if the numbers of records are enormous in the source table.

我们更新了源表中的一条记录，它应该只更新该特定记录。 它更新了所有记录，并且我们可能不想更新所有记录，尤其是如果源表中的记录数很大时。

Let’s modify the SSIS package to update only the required row in the destination table.

让我们修改SSIS包以仅更新目标表中的必需行。

Add another Lookup task in the Control flow and join it with the Lookup for the Lookup Match Output.

在控制流中添加另一个Lookup任务，并将其与Lookup Match Output的Lookup结合在一起。

Right-click on the Lookup1 and choose Redirect rows to no match output in the Specify how to handle rows with no matching rows.

右键单击Lookup1，然后在“ 指定如何处理不匹配的行”中选择“ 将行重定向到不匹配的输出” 。

In the Connection tab, use the destination table from the drop-down list.

在“连接”选项卡中，使用下拉列表中的目标表。

In the mapping, create the mapping for the input and Lookup column. You can right-click on the mapping screen and create Relationships for all columns similar to the following.

在映射中，为输入和查找列创建映射。 您可以在映射屏幕上单击鼠标右键，然后为所有列创建“关系”，类似于以下内容。

After the mapping is done, you can see the following screen in the Lookup Transformation Editor.

映射完成后，您可以在“查找转换编辑器”中看到以下屏幕。

In the next step, join the Lookup1 with the Update Records we configured earlier.

在下一步中，将Lookup1与我们之前配置的更新记录一起。

Once we try to join both the tasks, it opens the Input-Output selection.

一旦我们尝试同时加入这两个任务，它将打开Input-Output选择。

• Output: Lookup No Match Output 输出：查找无匹配输出
• Input: OLE DB Command Input 输入：OLE DB命令输入

Click Ok, and we see the following configured SSIS package.

单击确定，我们将看到以下已配置的SSIS程序包。

Execute the following update statement to change the book price for the [BookID] 1. This command updates a single record in the source table.

执行以下更新语句以更改[BookID] 1的书价。此命令更新源表中的一条记录。

UPDATE dbo.booksSET [Book Price] = 390WHERE BookID = 1;

Our SSIS package should also update one record only in the destination table. Execute the package. In the following screenshot, note down the following.

我们的SSIS包还应该只更新目标表中的一条记录。 执行包。 在以下屏幕截图中，记下以下内容。

• It does not insert any row in the destination table because we did not insert any new row in the source table 它不会在目标表中插入任何行，因为我们没有在源表中插入任何新行
• It updates only a single row in the destination table. It is in line with the updates we performed earlier 它仅更新目标表中的一行。 这与我们之前执行的更新是一致的

Let’s execute the following queries to insert a new record and update an existing record as well.

让我们执行以下查询以插入新记录并更新现有记录。

UPDATE dbo.booksSET [Book Price] = 750WHERE BookID = 2;goInsert into dbo.books values (8,'SSIS package',987)

Re-execute the SSIS package, and you can see it inserted 1 row in the destination table and updated a single row as well.

重新执行SSIS包，您可以看到它在目标表中插入了1行，并且也更新了一行。

Query both the source and destination tables, and verify the results. We see both the updates and inserted rows in the destination table.

查询源表和目标表，并验证结果。 我们在目标表中看到更新和插入的行。

结论 (Conclusion)

In this article, we covered Lookup Transformation in SSIS for incremental loading of data in the destination tables. It is a nice feature transformation in the SSIS, and you should know of it.

在本文中，我们介绍了SSIS中的查找转换，用于在目标表中增量加载数据。 这是SSIS中很好的功能转换，您应该知道这一点。

翻译自: https://www.sqlshack.com/an-overview-of-the-lookup-transformation-in-ssis/

ssis lookup