Showing posts with label package. Show all posts
Showing posts with label package. Show all posts

Tuesday, March 27, 2012

Creation of "global" variables in runtime.

Is it possible to create "global" (package scope) variables in runtime in SSIS packages? If yes, please give some examples.

Hi Yuriy:

Not sure I understand your question. Are you asking how to create variables that are consumed at runtime, or to have on-the-fly variables created as a package runs?

The first type is easy -- go to the Control Flow tab, make sure nothing is selected, and right-click anywhere in the white space outside of an object, and select "Variables" from the shortcut menu. Alternatively, select "Variables" from the SSIS menu at the top of the main window.

The dialog you see is pretty self-explanatory. Click on the blue "x" button in the toolbar to add a new variable. Give it a name, set its data type and initial value. Delete a variable by selecting it in the list and clicking the X button with the red "x" over it.

Once you create a variable it can be referenced through code in a Script Task or expressions in other tasks. Most often you must reference the variable as @.[User::<VariableNameHere>]. Notice the two colons (::) between "User" and the variable name, and the @. sign in front of the expression.

If an object in the Control Flow tab is selected when you invoke the Variables command, the variable is local to that object. I do this now and then on containers when I need a variable that is used to share data among the tasks within a container.

I don't believe it's possible to create entirely new variables at runtime.

|||No object can be created at runtime inside the packge.

Monday, March 19, 2012

creating SSIS packages in c#

How can I create a SSIS package using C#? Can anybody have source code for the same.

thanx in advance,

Vinod

You can use the object model of SSIS to do the same. Please refer to the books online for the SSIS Object Model.

Thanks,

S Suresh

Creating SSIS Package to Extract Data from MySQL and Append to SQL 2005

Hi guys,

I'm a newbie DBA and i'm trying to create a package that would extract data from MySQL and inserts them to a SQL 2005 Server. I'm quite new to this SSIS and would like to ask help from you to help me go through with this.

I hope you guys can help me with this.

Hoping to hear from you soon.

Thank you so much.

Kind regards,
Neil
I'm sure everyone on the forum will be happy to help. I'd suggest starting by going through the SSIS Tutorials in SQL Server Books Online. Then, as you encounter specific problems or design questions, we'll be happy to answer them.|||

Yep, I agree with John.

Just a hint though, you'll be using a DataReader Source against your MySQL ODBC driver to retrieve the results.

|||

Actually, i have gone through the BOL tutorial and have started creating the SSIS package using DataReader Source. i then provided the connection manager which is in my case using ADO.Net connection (MySQL ODBC 3.51 Driver) but as soon as i enter the SQLCommand and click ok this error displays on the bottom part of the Advanced Editor for DataReader Source.

Error at Data Flow Task [DataReader Source[1]]: System.Data.Odbc.OdbcException: ERROR [HY010]

[MySQL][ODBC 3.51 Driver][mysqld-4.1.12-Debian_1ubuntu3.4-log]

What do u think is causing this?

Thanks,

Neil


|||

If you could provide the query, that would help troubleshoot the issue.

If you are trying to use parameters or a complex query, try building it in a variable and using an expression on the data flow to set the SQL source. Expressions for the data flow can be found by clicking the data flow, opeing the properties window (F4), and looking for the expressions item.

|||

Below is the SQLCommand i am using. What's troubling me is that the DTS for this in SQL 2000 is running smoothly that is why i was thinking maybe i am using the wrong steps in SSIS.

SELECT chatrequestlogs.`chat_session`,
chat_admin.`name`,
chatrequestlogs.`ip`,
FROM_UNIXTIME(chatrequestlogs.created) AS Time_Requested,
FROM_UNIXTIME(chattranscripts.created) AS Time_Created,
chatrequestlogs.status,
LTRIM(substring(chattranscripts.plain, 1, locate(':',chattranscripts.plain, 1) - 1)) AS Name_Registered,
LTRIM(substring(substring(chattranscripts.formatted, locate('<question>', chattranscripts.formatted,1), locate('</question>', chattranscripts.formatted, locate('<question>', chattranscripts.formatted, 1)) - locate('<question>', chattranscripts.formatted, 1)),11)) AS Question,
LTRIM(substring(chattranscripts.plain, locate(':', chattranscripts.plain, 1) + 1)) AS Comments,
chatrequestlogs.deptID,
substring(substring(formatted, locate('<tstamp (', formatted), locate(') tstamp>', formatted) - locate('<tstamp (', formatted)), 10) AS Time_Started,
reverse(substring(substring(reverse(formatted), locate('>pmatst )', reverse(formatted)), locate('( pmatst<', reverse(formatted)) - locate('>pmatst )', reverse(formatted))), 10)) AS Time_Ended,
chat_admin.`email` AS CSCEmail,
chattranscripts.`email` AS CustomerEmail
FROM chatrequestlogs
JOIN chat_admin ON chatrequestlogs.userID = chat_admin.`userID`
JOIN chattranscripts ON chattranscripts.chat_session = chatrequestlogs.chat_session
WHERE FROM_UNIXTIME(chatrequestlogs.created) >= DATE_ADD(NOW(), INTERVAL - '1440' MINUTE)
AND FROM_UNIXTIME(chatrequestlogs.created) < NOW()

|||

Hi guys,

The problem went away after I turned the ValidateExternalMetadata property of the datasource reader to false.

Thank you so much for the replies.

Forums like these are of great help to newbies like me.

Keep up the good work guys.

Kind regards,

Creating SSIS package failed

Hi,

I'm trying to create a SSIS package. I have used OLEDBSource adapter to get the source table's data and transferring the data to an OLEDBDestination adapter. I tried but I'm facing the problem in mapping the metadata contents.

My code:

//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

static void Main(string[] args)

{

// Create a new package

Package package = new Package();

package.Name = "OLE DB Transfer";

// Add a Data Flow task

TaskHost taskHost = package.Executables.Add("DTS.Pipeline") as TaskHost;

taskHost.Name = "Transfer Table";

IDTSPipeline90 pipeline = taskHost.InnerObject as MainPipe;

// Get the pipeline's component metadata collection

IDTSComponentMetaDataCollection90 componentMetadataCollection = pipeline.ComponentMetaDataCollection;

// Add a new component metadata object to the data flow

IDTSComponentMetaData90 oledbSourceMetadata = componentMetadataCollection.New();

// Associate the component metadata object with the OLE DB Source Adapter

oledbSourceMetadata.ComponentClassID = "DTSAdapter.OLEDBSource";

// Instantiate the OLE DB Source adapter

IDTSDesigntimeComponent90 oledbSourceComponent = oledbSourceMetadata.Instantiate();

// Ask the component to set up its component metadata object

oledbSourceComponent.ProvideComponentProperties();

// Add an OLE DB connection manager

ConnectionManager connectionManagerSource = package.Connections.Add("OLEDB");

connectionManagerSource.Name = "OLEDBSource";

// Set the connection string

connectionManagerSource.ConnectionString = "provider=sqlncli;server=HSCHBSCGN25008;integrated security=sspi;database=Muthu_SSIS_Testing";

// Set the connection manager as the OLE DB Source adapter's runtime connection

IDTSRuntimeConnection90 runtimeConnectionSource = oledbSourceMetadata.RuntimeConnectionCollection["OleDbConnection"];

runtimeConnectionSource.ConnectionManagerID = connectionManagerSource.ID;

// Tell the OLE DB Source adapter to use the SQL Command access mode.

oledbSourceComponent.SetComponentProperty("AccessMode", 2);

// Set up the SQL command

oledbSourceComponent.SetComponentProperty("SqlCommand", "select * from EmployeeTable");

// Set up the connection manager object

runtimeConnectionSource.ConnectionManager = DtsConvert.ToConnectionManager90(connectionManagerSource);

// Establish the database connection

oledbSourceComponent.AcquireConnections(null);

// Set up the column metadata

oledbSourceComponent.ReinitializeMetaData();

// Release the database connection

oledbSourceComponent.ReleaseConnections();

// Release the connection manager

runtimeConnectionSource.ReleaseConnectionManager();

// Add a new component metadata object to the data flow

IDTSComponentMetaData90 oledbDestinationMetadata = componentMetadataCollection.New();

//Associate the component metadata object with the OLE DB Destination Adapter

oledbDestinationMetadata.ComponentClassID = "DTSAdapter.OLEDBDestination";

// Instantiate the OLE DB Destination adapter

IDTSDesigntimeComponent90 oledbDestinationComponent = oledbDestinationMetadata.Instantiate();

// Ask the component to set up its component metadata object

oledbDestinationComponent.ProvideComponentProperties();

// Add an OLE DB connection manager

ConnectionManager connectionManagerDestination = package.Connections.Add("OLEDB");

connectionManagerDestination.Name = "OLEDBDestination";

// Set the connection string

connectionManagerDestination.ConnectionString = "provider=sqlncli;server=HSCHBSCGN25008;integrated security=sspi;database=Muthu_SSIS_Testing";

// Set the connection manager as the OLE DB Destination adapter's runtime connection

IDTSRuntimeConnection90 runtimeConnectionDestination = oledbDestinationMetadata.RuntimeConnectionCollection["OleDbConnection"];

runtimeConnectionDestination.ConnectionManagerID = connectionManagerDestination.ID;

// Tell the OLE DB Destination adapter to use the SQL Command access mode.

oledbDestinationComponent.SetComponentProperty("AccessMode", 2);

// Set up the SQL command

oledbDestinationComponent.SetComponentProperty("SqlCommand", "select from EmplTable");

// Set up the connection manager object

runtimeConnectionDestination.ConnectionManager = DtsConvert.ToConnectionManager90(connectionManagerDestination);

// Get the standard output of the OLE DB Source adapter

IDTSOutput90 oledbSourceOutput = oledbSourceMetadata.OutputCollection["OLE DB Source Output"];

// Get the input of the OLE DB Destination adapter

IDTSInput90 oledbDestinationInput = oledbDestinationMetadata.InputCollection["OLE DB Destination Input"];

// Create a new path object

IDTSPath90 path = pipeline.PathCollection.New();

// Connect the source and destination adapters

path.AttachPathAndPropagateNotifications(oledbSourceOutput, oledbDestinationInput);

IDTSInput90 input = oledbDestinationInput;

IDTSVirtualInput90 vInput = input.GetVirtualInput();

foreach (IDTSVirtualInputColumn90 vColumn in vInput.VirtualInputColumnCollection)

{

// Call the SetUsageType method of the destination

// to add each available virtual input column as an input column.

oledbDestinationComponent.SetUsageType(

input.ID, vInput, vColumn.LineageID, DTSUsageType.UT_READONLY);

}

foreach (IDTSInputColumn90 col in oledbDestinationInput.InputColumnCollection)

{

IDTSExternalMetadataColumn90 exCol = oledbDestinationInput.ExternalMetadataColumnCollection[col.Name];

oledbDestinationComponent.MapInputColumn(oledbDestinationInput.ID, col.ID, exCol.ID);

}

Console.WriteLine("done");

Console.ReadKey();

// Save the package

//Application application = new Application();

//application.SaveToXml(@."c:\OLEDBTransfer.dtsx", package, null);

}

While debugging, I'm getting exception (ELEMENTNOTFOUND) at this line

IDTSExternalMetadataColumn90 exCol = oledbDestinationInput.ExternalMetadataColumnCollection[col.Name];

because metadata have no appropriate column to map with the destination component.

Any one help me to resolve this issue.

Regards,

kris

Hi,

I have resolved my issues.

For destination component, I initialized the connection and reinitialized metadata and done the mapping then released the connection for destination component.

Now my code is working.

Thanks for your helps..

Regards,

kris

Creating SSIS Package Event Handlers

I would like to create an event handler that would catch any errors that result from a sys.<table> not existing. The package is designed to run on both SQL Server 2000 and SQL Server 2005 and when I query sys.<tables> there is an error when the query is run on SQL Server 2000. I just need a good starting point...I would like something that when the server isn't 2005 it just skips the server and doesn't fail the package and doesn't get counted towards the max error count. Thanks for any help.
-Kyle

Add an Execute SQL task that queries @.@.VERSION prior to the query that uses the sys.*. Then you can use precedence constraints based on the version value to determine whether the tasks execute.

Code Snippet

SELECT @.@.VERSION

Creating sql server table from dts

I am trying to design a SSIS DTS package that imports an Excel named range (acts as a table) into a SQL Server 2005 database table. The direct import (using the import/export wizard) works fine, but I need to use a DTS package with a data transformation step. The DTS fails to create/replace/or update the SQL Server table. Suggestions?

Which is it? SSIS or DTS?

Can you clarify what you're trying to do?

How are you attempting to create/replace/update the table?

Kirk Haselden
Author "SQL Server Integration Services"

|||

Kirk, Thank you for the response. Your book will arrive tomorrow and I expect it to answer my many questions. I am trying to use the SQL Server 2005 SSIS designer to produce a package (SSIS or DTS?) to copy a named range (proxy for an Excel 'table') from a network drive; convert the data types to what I want on the SQL Server 2005 server; delete the database table if it already exists and create a new table; and write the new data to the table. The connection to the Excel workbook named range and the data conversion step work fine. I get the following error in the destination step: [SQL Server Destination [98]] Error: An OLE DB error has occurred. Error code: 0x80040E14. An OLE DB record is available. Source: "Microsoft SQL Native Client" Hresult: 0x80040E14 Description: "Could not bulk load because SSIS file mapping object 'Global\DTSQLIMPORT ' could not be opened. Operating system error code 2(The system cannot find the file specified.). Make sure you are accessing a local server via Windows security." I have no idea what this means!

I am a 'user' rather than an 'IT pro' who is trying to build a small 'data-mart' for better management of data used for analysis, forecasting and strategic planning. A SSIS package appears to be the best means for accomplishing this task every month and many other data upload packages will be produced as soon as I understand the process.

Thank you for your help.

|||Kirk Haselden's book on Integration Services is very helpful. I highly recommend it!

Creating sql server table from dts

I am trying to design a SSIS DTS package that imports an Excel named range (acts as a table) into a SQL Server 2005 database table. The direct import (using the import/export wizard) works fine, but I need to use a DTS package with a data transformation step. The DTS fails to create/replace/or update the SQL Server table. Suggestions?

Which is it? SSIS or DTS?

Can you clarify what you're trying to do?

How are you attempting to create/replace/update the table?

Kirk Haselden
Author "SQL Server Integration Services"

|||

Kirk, Thank you for the response. Your book will arrive tomorrow and I expect it to answer my many questions. I am trying to use the SQL Server 2005 SSIS designer to produce a package (SSIS or DTS?) to copy a named range (proxy for an Excel 'table') from a network drive; convert the data types to what I want on the SQL Server 2005 server; delete the database table if it already exists and create a new table; and write the new data to the table. The connection to the Excel workbook named range and the data conversion step work fine. I get the following error in the destination step: [SQL Server Destination [98]] Error: An OLE DB error has occurred. Error code: 0x80040E14. An OLE DB record is available. Source: "Microsoft SQL Native Client" Hresult: 0x80040E14 Description: "Could not bulk load because SSIS file mapping object 'Global\DTSQLIMPORT ' could not be opened. Operating system error code 2(The system cannot find the file specified.). Make sure you are accessing a local server via Windows security." I have no idea what this means!

I am a 'user' rather than an 'IT pro' who is trying to build a small 'data-mart' for better management of data used for analysis, forecasting and strategic planning. A SSIS package appears to be the best means for accomplishing this task every month and many other data upload packages will be produced as soon as I understand the process.

Thank you for your help.

|||Kirk Haselden's book on Integration Services is very helpful. I highly recommend it!

Saturday, February 25, 2012

Creating New Packages - Saving in Project Folder Unnecessarily

In BI Dev Studio, I have a Project/Solution created successfully, which has got say 5 packages working fine. I created a new package by Right Clicking on a solution explorer and selecting "New SSIS Package". System has given itself a default name and I decided to leave the BI Dev Studio WITHOUT SAVING project / solution.

Even though I haven't saved the project / solution still it left a package in project folder with a name "Package1.dtsx". I think this is a bug? Any ideas please?

Thanks

SuthaThis is true. I am sure this is a bug.Tongue Tied|||If it is a bug, it is one common to all VS products, and has been since VS .Net came out. The IDE will always create files on disk first. You must supply the filename upfront for a solution, project or file, although in the later case in can be inferred as it is here. I'd rather it save it for me, than me forget and loose work. VS will also save the package every time you execute it as well.|||Daran, thanks.

I understand your argument. I suppose if it is unnecessary we can delete manaually, rather than losing hours of work incase of any disaster. Tongue Tied

Friday, February 17, 2012

Creating File

Is there a corresponding utility in SQL Server like
UTIL_FILE package in ORacle which can be used to create
text files from Stored procs...Thanks in advance for helpYes, try with
execute xp_cmdshell 'echo blahblahblah > c:\test.txt'
--
Regards,
Tomislav Kralj
tomislav.kralj1@.zg.tel.hr
"Kris" <maran123us@.hotmail.com> wrote in message
news:00a701c34ce1$e1e994c0$a001280a@.phx.gbl...
> Is there a corresponding utility in SQL Server like
> UTIL_FILE package in ORacle which can be used to create
> text files from Stored procs...Thanks in advance for help

Creating excel file from DTS package

Hi,

I have a created a DTS packges which is reading data from sql server table, manipulate this data as required and then create a text file with that data. I created the text file using FileSystemObject. I was writing one field at a time to the text file.

I need to same thing but instead of creating text file, I need to create a excel file with each column from database going to separate column in excel sheet. I tried to do this with FileSystemObject, but it was wrting all the columns from database to one cell in excel sheet. How can I fix this problem?

Thanks!Am I missing something or can't you just create a data transformation task going from your database to a excel spreadsheet?

Create a conenction to a spreadsheet and a connection to a database and build your transformation. DTS will take care of everything else. If you need to play with filesnames etc you can do that after the package has been created (using the filesystem object)|||is there any sample to show that how to ceate an excel file after reading data from sql server?

Thanks|||start>programs>Microsoft SQL Server>Import and Export Data

follow the wizard. select your db>Select your output file (excel)>hit next>run immediately>done|||Sorry, I didn't explain my question properly. I can create an excel file and write data after reading from database but I also need to write totals, tax and grand totals after I finish writing all the records from the database. How can I add totals to the last line? I need to do all of this from a DTS package.

Thanks!|||I can create an excel file and write data after reading from database but I also need to write totals, tax and grand totals after I finish writing all the records from the database. How can I add totals to the last line? I need to do all of this from a DTS package.

Thanks!|||I 've create a template with the functions for addign totals, etc already built in. using VB SCRIPT, copy the template and populate it every time you run the DTS package.|||Could you please explain how can I create this template and copy it every time I need this?

Thanks|||'************************************************* *********************
' Visual Basic ActiveX Script
'************************************************* ***********************

Function Main()

Dim objFSO, objFileTemplate, strFile

Set objFSO = CreateObject ( "Scripting.FileSystemObject" )
strFile = DTSGlobalVariables("strRootPath").Value & "Template.xls"
Set objFileTemplate = objFSO.GetFile ( strFile )
strMonth = Right ( "0" & DatePart ( "m", DTSGlobalVariables("dtStart").Value ), 2 )
strFile = DTSGlobalVariables("strPath").Value & _
DatePart ( "yyyy", DTSGlobalVariables("dtStart").Value ) & _
strMonth & _
" - " & _
DTSGlobalVariables("ID").Value & " " & DTSGlobalVariables("Id").Value & _
".xls"
DTSGlobalVariables("FileName").Value = strFile

If ( objFSO.FileExists ( strFile ) ) Then
Main = DTSTaskExecResult_Failure
Exit Function
End If

objFileTemplate.Copy strFile, 0

Main = DTSTaskExecResult_Success

End Function

Note: This is basically the code. You have a template in a directory. This vbscrip takes the template, copies it to a different directory. The name is obtained by using the 'Id' field and concatenating month and year.

If you want, we can take this offline, just let me know your email address.

Creating DTS Package to archive database from one sql server to another

Hi...

I am new to database programming.

I need to create a DTS Package which should archive a database fron one sql server to another.

I have no idea that which connection objects and which task objects should be used in order to accomplish this task.

Please help me with this.

Any one of your help is greatly appreciated.

Thank youHi There,

Use "Transfer Databases" task. You can find that under Task menu.

Select appropriate options under each tab. Its pretty straight forward.

Test this DTS first using a test database before implementation.

Good Luck!|||I have moved your question from the Articles section of our site to the Forum.

Creating DTS

Are you referring to Data Transformation Services package in SQL Server 2000 or SQL Server Integration Services package in SQL Server 2005? If it's the latter you can create an SSIS project using the Business Intelligence Development Studio which is part of SQL Server 2005 in editions from Developer Edition and higher. Andrew Watt MVP - InfoPath wrote in message news:6a53eb64-e504-4710-99bf-527c9aa02e46@.discussions.microsoft.com...
> Is there a way to create a DTS from Windows application ? >
> And how can I send parameters and configure DTS from my CSharp Code ?
>Is there a way to create a DTS from Windows application ?

And how can I send parameters and configure DTS from my CSharp Code ?|||I'm talking about Data Transformation Services|||You should direct your question to the "SQL Server Integration Services " forum. That is where the DTS experts live.

Thanks|||Valew|||The public newsgroup for Data Transformation Services in SQL Server 2000 is microsoft.public.sqlserver.dts. Andrew Watt MVP - InfoPath wrote in message news:cec83e18-03a9-4b2c-95ed-d86199314175@.discussions.microsoft.com...
> I'm talking about Data Transformation Services
>

Tuesday, February 14, 2012

Creating Custom Sequence Container

I have a package that is going to have roughly 20 sequence containers in it. Each of these containers is going to have the same start task and the same end task. The data flow task(s) in each sequence container will differ. So I was thinking I would create a custom sequence container that would implement the common steps I need in each of my containers. I started to create a task that would inherit from Sequence, but I found that it is sealed. Bummer.

What is involved in creating a sequence task by inheriting from Task? I see that Sequence implements IDtsSequence. But looking at this interface, it seems like I would have to build the logic to execute the workflow contained in my sequence.

Any pointers?

-Darrell

Darrell,

What you're talking about here is task reuse right? So implementing a new type of container won't help you and you cannot do it anyway as far as I am aware.

The only unit of reuse in SSIS currently is a package so you will need to hive off that repeatable functionality into another package and call it using the Execute Package Task.

There are huge improvements to be made in the area of reusability and reuse of tasks rather packages is my numero uno for vNext. Read more here: http://blogs.conchango.com/jamiethomson/archive/2005/02/05/929.aspx

-Jamie

|||

I don't think I explained my issue well enough if you think that a new type of container won't help. I want to create a container that when it "starts" will update a status in a table and when it "ends" it will update the status again. I planned to use a custom property in my custom container object so that each instance of my container can differentiate itself in the status table. What my container "contains" would vary with each instance in the package. Hence, the control needs to be a container.

Bummer I can't make my own.

-Darrell

|||

Darrell Davis wrote:

I don't think I explained my issue well enough if you think that a new type of container won't help. I want to create a container that when it "starts" will update a status in a table and when it "ends" it will update the status again. I planned to use a custom property in my custom container object so that each instance of my container can differentiate itself in the status table. What my container "contains" would vary with each instance in the package. Hence, the control needs to be a container.

Bummer I can't make my own.

-Darrell

Containers in the context that we are talking about them do not and should not affect external data. That is not what they are for. They are to affect control-flow, nothing else.

Updating tables can be done very easily using the Execute SQL Task. Use the right tool for the job is I guess what I'm saying here.

If you want to achieve the thing that you are looking to achieve with what you refer to as a custom property, all you need to do is scope a variable to the Sequence container that holds that container's "ID". Each of your 20 sequence containers has an identically named variable scoped to it with a different value.

Hope that helps.

-Jamie

|||To be clear, sequence containers are not extensible, you cannot write your own. You could perhaps write the the start and end processes as task, that require minimal setup. This would make for faster development, and also encapsulat ethe logic in a reusable unit and also make them easy to maintain, that is just change the task code and redeploy, not maintain n packages or n instances of the task in a package.|||

Thanks, that's what I ended up doing. I created a new Task which has my start code and my end code. It has a property which indicates which MainPipe to execute in what would be the body of the sequence. It loads the TaskHost that contains the MainPipe at runtime and passes in the variables that are neccessary. It is not as elegant as it would be if I could subclass Sequence, but it seems to work pretty well for what I need. It reduces the number of tasks in my package from around 180 to 30. Also, as you poitned out, it allows me to modify my start and end code one place in my custom task and then just redeploy.

Thanks,
-Darrell