Showing posts with label ssis. Show all posts
Showing posts with label ssis. Show all posts

Tuesday, March 27, 2012

Creation of "global" variables in runtime.

Is it possible to create "global" (package scope) variables in runtime in SSIS packages? If yes, please give some examples.

Hi Yuriy:

Not sure I understand your question. Are you asking how to create variables that are consumed at runtime, or to have on-the-fly variables created as a package runs?

The first type is easy -- go to the Control Flow tab, make sure nothing is selected, and right-click anywhere in the white space outside of an object, and select "Variables" from the shortcut menu. Alternatively, select "Variables" from the SSIS menu at the top of the main window.

The dialog you see is pretty self-explanatory. Click on the blue "x" button in the toolbar to add a new variable. Give it a name, set its data type and initial value. Delete a variable by selecting it in the list and clicking the X button with the red "x" over it.

Once you create a variable it can be referenced through code in a Script Task or expressions in other tasks. Most often you must reference the variable as @.[User::<VariableNameHere>]. Notice the two colons (::) between "User" and the variable name, and the @. sign in front of the expression.

If an object in the Control Flow tab is selected when you invoke the Variables command, the variable is local to that object. I do this now and then on containers when I need a variable that is used to share data among the tasks within a container.

I don't believe it's possible to create entirely new variables at runtime.

|||No object can be created at runtime inside the packge.

Monday, March 19, 2012

creating SSIS packages in c#

How can I create a SSIS package using C#? Can anybody have source code for the same.

thanx in advance,

Vinod

You can use the object model of SSIS to do the same. Please refer to the books online for the SSIS Object Model.

Thanks,

S Suresh

Creating SSIS Package to Extract Data from MySQL and Append to SQL 2005

Hi guys,

I'm a newbie DBA and i'm trying to create a package that would extract data from MySQL and inserts them to a SQL 2005 Server. I'm quite new to this SSIS and would like to ask help from you to help me go through with this.

I hope you guys can help me with this.

Hoping to hear from you soon.

Thank you so much.

Kind regards,
Neil
I'm sure everyone on the forum will be happy to help. I'd suggest starting by going through the SSIS Tutorials in SQL Server Books Online. Then, as you encounter specific problems or design questions, we'll be happy to answer them.|||

Yep, I agree with John.

Just a hint though, you'll be using a DataReader Source against your MySQL ODBC driver to retrieve the results.

|||

Actually, i have gone through the BOL tutorial and have started creating the SSIS package using DataReader Source. i then provided the connection manager which is in my case using ADO.Net connection (MySQL ODBC 3.51 Driver) but as soon as i enter the SQLCommand and click ok this error displays on the bottom part of the Advanced Editor for DataReader Source.

Error at Data Flow Task [DataReader Source[1]]: System.Data.Odbc.OdbcException: ERROR [HY010]

[MySQL][ODBC 3.51 Driver][mysqld-4.1.12-Debian_1ubuntu3.4-log]

What do u think is causing this?

Thanks,

Neil


|||

If you could provide the query, that would help troubleshoot the issue.

If you are trying to use parameters or a complex query, try building it in a variable and using an expression on the data flow to set the SQL source. Expressions for the data flow can be found by clicking the data flow, opeing the properties window (F4), and looking for the expressions item.

|||

Below is the SQLCommand i am using. What's troubling me is that the DTS for this in SQL 2000 is running smoothly that is why i was thinking maybe i am using the wrong steps in SSIS.

SELECT chatrequestlogs.`chat_session`,
chat_admin.`name`,
chatrequestlogs.`ip`,
FROM_UNIXTIME(chatrequestlogs.created) AS Time_Requested,
FROM_UNIXTIME(chattranscripts.created) AS Time_Created,
chatrequestlogs.status,
LTRIM(substring(chattranscripts.plain, 1, locate(':',chattranscripts.plain, 1) - 1)) AS Name_Registered,
LTRIM(substring(substring(chattranscripts.formatted, locate('<question>', chattranscripts.formatted,1), locate('</question>', chattranscripts.formatted, locate('<question>', chattranscripts.formatted, 1)) - locate('<question>', chattranscripts.formatted, 1)),11)) AS Question,
LTRIM(substring(chattranscripts.plain, locate(':', chattranscripts.plain, 1) + 1)) AS Comments,
chatrequestlogs.deptID,
substring(substring(formatted, locate('<tstamp (', formatted), locate(') tstamp>', formatted) - locate('<tstamp (', formatted)), 10) AS Time_Started,
reverse(substring(substring(reverse(formatted), locate('>pmatst )', reverse(formatted)), locate('( pmatst<', reverse(formatted)) - locate('>pmatst )', reverse(formatted))), 10)) AS Time_Ended,
chat_admin.`email` AS CSCEmail,
chattranscripts.`email` AS CustomerEmail
FROM chatrequestlogs
JOIN chat_admin ON chatrequestlogs.userID = chat_admin.`userID`
JOIN chattranscripts ON chattranscripts.chat_session = chatrequestlogs.chat_session
WHERE FROM_UNIXTIME(chatrequestlogs.created) >= DATE_ADD(NOW(), INTERVAL - '1440' MINUTE)
AND FROM_UNIXTIME(chatrequestlogs.created) < NOW()

|||

Hi guys,

The problem went away after I turned the ValidateExternalMetadata property of the datasource reader to false.

Thank you so much for the replies.

Forums like these are of great help to newbies like me.

Keep up the good work guys.

Kind regards,

Creating SSIS package failed

Hi,

I'm trying to create a SSIS package. I have used OLEDBSource adapter to get the source table's data and transferring the data to an OLEDBDestination adapter. I tried but I'm facing the problem in mapping the metadata contents.

My code:

//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

static void Main(string[] args)

{

// Create a new package

Package package = new Package();

package.Name = "OLE DB Transfer";

// Add a Data Flow task

TaskHost taskHost = package.Executables.Add("DTS.Pipeline") as TaskHost;

taskHost.Name = "Transfer Table";

IDTSPipeline90 pipeline = taskHost.InnerObject as MainPipe;

// Get the pipeline's component metadata collection

IDTSComponentMetaDataCollection90 componentMetadataCollection = pipeline.ComponentMetaDataCollection;

// Add a new component metadata object to the data flow

IDTSComponentMetaData90 oledbSourceMetadata = componentMetadataCollection.New();

// Associate the component metadata object with the OLE DB Source Adapter

oledbSourceMetadata.ComponentClassID = "DTSAdapter.OLEDBSource";

// Instantiate the OLE DB Source adapter

IDTSDesigntimeComponent90 oledbSourceComponent = oledbSourceMetadata.Instantiate();

// Ask the component to set up its component metadata object

oledbSourceComponent.ProvideComponentProperties();

// Add an OLE DB connection manager

ConnectionManager connectionManagerSource = package.Connections.Add("OLEDB");

connectionManagerSource.Name = "OLEDBSource";

// Set the connection string

connectionManagerSource.ConnectionString = "provider=sqlncli;server=HSCHBSCGN25008;integrated security=sspi;database=Muthu_SSIS_Testing";

// Set the connection manager as the OLE DB Source adapter's runtime connection

IDTSRuntimeConnection90 runtimeConnectionSource = oledbSourceMetadata.RuntimeConnectionCollection["OleDbConnection"];

runtimeConnectionSource.ConnectionManagerID = connectionManagerSource.ID;

// Tell the OLE DB Source adapter to use the SQL Command access mode.

oledbSourceComponent.SetComponentProperty("AccessMode", 2);

// Set up the SQL command

oledbSourceComponent.SetComponentProperty("SqlCommand", "select * from EmployeeTable");

// Set up the connection manager object

runtimeConnectionSource.ConnectionManager = DtsConvert.ToConnectionManager90(connectionManagerSource);

// Establish the database connection

oledbSourceComponent.AcquireConnections(null);

// Set up the column metadata

oledbSourceComponent.ReinitializeMetaData();

// Release the database connection

oledbSourceComponent.ReleaseConnections();

// Release the connection manager

runtimeConnectionSource.ReleaseConnectionManager();

// Add a new component metadata object to the data flow

IDTSComponentMetaData90 oledbDestinationMetadata = componentMetadataCollection.New();

//Associate the component metadata object with the OLE DB Destination Adapter

oledbDestinationMetadata.ComponentClassID = "DTSAdapter.OLEDBDestination";

// Instantiate the OLE DB Destination adapter

IDTSDesigntimeComponent90 oledbDestinationComponent = oledbDestinationMetadata.Instantiate();

// Ask the component to set up its component metadata object

oledbDestinationComponent.ProvideComponentProperties();

// Add an OLE DB connection manager

ConnectionManager connectionManagerDestination = package.Connections.Add("OLEDB");

connectionManagerDestination.Name = "OLEDBDestination";

// Set the connection string

connectionManagerDestination.ConnectionString = "provider=sqlncli;server=HSCHBSCGN25008;integrated security=sspi;database=Muthu_SSIS_Testing";

// Set the connection manager as the OLE DB Destination adapter's runtime connection

IDTSRuntimeConnection90 runtimeConnectionDestination = oledbDestinationMetadata.RuntimeConnectionCollection["OleDbConnection"];

runtimeConnectionDestination.ConnectionManagerID = connectionManagerDestination.ID;

// Tell the OLE DB Destination adapter to use the SQL Command access mode.

oledbDestinationComponent.SetComponentProperty("AccessMode", 2);

// Set up the SQL command

oledbDestinationComponent.SetComponentProperty("SqlCommand", "select from EmplTable");

// Set up the connection manager object

runtimeConnectionDestination.ConnectionManager = DtsConvert.ToConnectionManager90(connectionManagerDestination);

// Get the standard output of the OLE DB Source adapter

IDTSOutput90 oledbSourceOutput = oledbSourceMetadata.OutputCollection["OLE DB Source Output"];

// Get the input of the OLE DB Destination adapter

IDTSInput90 oledbDestinationInput = oledbDestinationMetadata.InputCollection["OLE DB Destination Input"];

// Create a new path object

IDTSPath90 path = pipeline.PathCollection.New();

// Connect the source and destination adapters

path.AttachPathAndPropagateNotifications(oledbSourceOutput, oledbDestinationInput);

IDTSInput90 input = oledbDestinationInput;

IDTSVirtualInput90 vInput = input.GetVirtualInput();

foreach (IDTSVirtualInputColumn90 vColumn in vInput.VirtualInputColumnCollection)

{

// Call the SetUsageType method of the destination

// to add each available virtual input column as an input column.

oledbDestinationComponent.SetUsageType(

input.ID, vInput, vColumn.LineageID, DTSUsageType.UT_READONLY);

}

foreach (IDTSInputColumn90 col in oledbDestinationInput.InputColumnCollection)

{

IDTSExternalMetadataColumn90 exCol = oledbDestinationInput.ExternalMetadataColumnCollection[col.Name];

oledbDestinationComponent.MapInputColumn(oledbDestinationInput.ID, col.ID, exCol.ID);

}

Console.WriteLine("done");

Console.ReadKey();

// Save the package

//Application application = new Application();

//application.SaveToXml(@."c:\OLEDBTransfer.dtsx", package, null);

}

While debugging, I'm getting exception (ELEMENTNOTFOUND) at this line

IDTSExternalMetadataColumn90 exCol = oledbDestinationInput.ExternalMetadataColumnCollection[col.Name];

because metadata have no appropriate column to map with the destination component.

Any one help me to resolve this issue.

Regards,

kris

Hi,

I have resolved my issues.

For destination component, I initialized the connection and reinitialized metadata and done the mapping then released the connection for destination component.

Now my code is working.

Thanks for your helps..

Regards,

kris

Creating SSIS Package Event Handlers

I would like to create an event handler that would catch any errors that result from a sys.<table> not existing. The package is designed to run on both SQL Server 2000 and SQL Server 2005 and when I query sys.<tables> there is an error when the query is run on SQL Server 2000. I just need a good starting point...I would like something that when the server isn't 2005 it just skips the server and doesn't fail the package and doesn't get counted towards the max error count. Thanks for any help.
-Kyle

Add an Execute SQL task that queries @.@.VERSION prior to the query that uses the sys.*. Then you can use precedence constraints based on the version value to determine whether the tasks execute.

Code Snippet

SELECT @.@.VERSION

Creating sql server table from dts

I am trying to design a SSIS DTS package that imports an Excel named range (acts as a table) into a SQL Server 2005 database table. The direct import (using the import/export wizard) works fine, but I need to use a DTS package with a data transformation step. The DTS fails to create/replace/or update the SQL Server table. Suggestions?

Which is it? SSIS or DTS?

Can you clarify what you're trying to do?

How are you attempting to create/replace/update the table?

Kirk Haselden
Author "SQL Server Integration Services"

|||

Kirk, Thank you for the response. Your book will arrive tomorrow and I expect it to answer my many questions. I am trying to use the SQL Server 2005 SSIS designer to produce a package (SSIS or DTS?) to copy a named range (proxy for an Excel 'table') from a network drive; convert the data types to what I want on the SQL Server 2005 server; delete the database table if it already exists and create a new table; and write the new data to the table. The connection to the Excel workbook named range and the data conversion step work fine. I get the following error in the destination step: [SQL Server Destination [98]] Error: An OLE DB error has occurred. Error code: 0x80040E14. An OLE DB record is available. Source: "Microsoft SQL Native Client" Hresult: 0x80040E14 Description: "Could not bulk load because SSIS file mapping object 'Global\DTSQLIMPORT ' could not be opened. Operating system error code 2(The system cannot find the file specified.). Make sure you are accessing a local server via Windows security." I have no idea what this means!

I am a 'user' rather than an 'IT pro' who is trying to build a small 'data-mart' for better management of data used for analysis, forecasting and strategic planning. A SSIS package appears to be the best means for accomplishing this task every month and many other data upload packages will be produced as soon as I understand the process.

Thank you for your help.

|||Kirk Haselden's book on Integration Services is very helpful. I highly recommend it!

Creating sql server table from dts

I am trying to design a SSIS DTS package that imports an Excel named range (acts as a table) into a SQL Server 2005 database table. The direct import (using the import/export wizard) works fine, but I need to use a DTS package with a data transformation step. The DTS fails to create/replace/or update the SQL Server table. Suggestions?

Which is it? SSIS or DTS?

Can you clarify what you're trying to do?

How are you attempting to create/replace/update the table?

Kirk Haselden
Author "SQL Server Integration Services"

|||

Kirk, Thank you for the response. Your book will arrive tomorrow and I expect it to answer my many questions. I am trying to use the SQL Server 2005 SSIS designer to produce a package (SSIS or DTS?) to copy a named range (proxy for an Excel 'table') from a network drive; convert the data types to what I want on the SQL Server 2005 server; delete the database table if it already exists and create a new table; and write the new data to the table. The connection to the Excel workbook named range and the data conversion step work fine. I get the following error in the destination step: [SQL Server Destination [98]] Error: An OLE DB error has occurred. Error code: 0x80040E14. An OLE DB record is available. Source: "Microsoft SQL Native Client" Hresult: 0x80040E14 Description: "Could not bulk load because SSIS file mapping object 'Global\DTSQLIMPORT ' could not be opened. Operating system error code 2(The system cannot find the file specified.). Make sure you are accessing a local server via Windows security." I have no idea what this means!

I am a 'user' rather than an 'IT pro' who is trying to build a small 'data-mart' for better management of data used for analysis, forecasting and strategic planning. A SSIS package appears to be the best means for accomplishing this task every month and many other data upload packages will be produced as soon as I understand the process.

Thank you for your help.

|||Kirk Haselden's book on Integration Services is very helpful. I highly recommend it!

Wednesday, March 7, 2012

Creating OLEDB Connection from SSIS to Inforrmix

Hi,

I am trying to Create an OLEDB Connection from SSIS to Inforrmix without any success. Would someone be able to help.We are running Informix version 9.4. We were able to create an OLEDB connection with SQL server 2000 and DTS. But, when trying to create an OLEDB connection using SQL server 2005 and SSIS, I am getting an error message. The connection is created successfully but cannot access databases and tables on informix. The error message when trying to load the Informix tables:

'Could not retrieve table information for the connection manager ....

'Ifxoledbc.e' failed with no error message available '

Your feedback is appreciated

Thank you,

Patrick.

Patrick

What environment is your Informix DB running in?

Shaun

Friday, February 24, 2012

creating jobs in SSIS

what is the equiv to dtsrunui to create the operating system (CmdExec)...
anyone have a good site suggestions for ssis outside of bolMaybe someone can help you with this, but I can't understand what you are asking.

-PatP|||I appreciate it ... i found what the equivalent is...
dtsrunui = dtexecui

Sunday, February 19, 2012

Creating GUI with SSIS or Passing parameter(s) in SSIS

Hello All!

I have two questions to ask in this one thread. I would appreciate any feedback.

1. Is it possible to create GUI from SSIS using macro so that it can display forms or dialogs? If so how can I create a form that can be used to pass the parameters for the execution of the SSIS?

2. Is it possible to pass parameter(s) to SSIS? If yes, how can we do it...Please provide me with any example.

I wait to hear from you!

Thanks,
Niben

1. Yes, you can use the Script task to show forms and gather user input. But you shouldn't. See #2 for information on the better approach

2. You can set values in an SSIS package from an external source in several ways. Configurations are an option that allow you to store values in an XML file or database table. These values are read in at run-time. These are used primarily for things like connection strings, and other relatively static information that may change once in a while, or when you move from test to production.

You can also set the value of a variable or property of the package when you execute it by using the /SET option of DTEXEC. This method is preferred for values that change each time the package is executed. In the case that you need user input for your package, I would suggest gathering it up front by creating a GUI in your choice of languages, then pass the values collected to the SSIS package using the /SET command.

Friday, February 17, 2012

Creating dynamic reports

Hello.

I wanted to know a way to create dynamic reports using reporting / analysis services in SQL Server 2005 / SSIS tool.

I want user to give inputs and generate reports based on those inputs.

Please guide me on this issue.

Thank you.

Regards,

Prathamesh

Hello Domnick....

In Report Services, you have basically two ways to do this....

1- The Input Paremeter can stay in the Report Services

2- The Input Parameter can stay in application and can be sent to Report Services by the Url ( Named Url Access )

1--> If in Query of your DataSet, you have a parameter (Ex: @.ID), Automatically it will be detected by Report Services, and it can be edited by the Report Parematers -->In Layout View / Right Click at the Fund Plane of Your Report / Report Parameters

Or the input parameter can be added manually in this same location (I don′t recomended)

2--> Sample :

http://server/reportserver?/Sales/Northwest/Employee Sales Report&rs:Command=Render&ID=1234

Is this your problem?