Enterprise Password Managers

May 10, 2018 Root Leave a comment

Below is a list of enterprise password managers

https://www.vaultproject.io/
HashiCorp Vault secures, stores, and tightly controls access to tokens, passwords, certificates, API keys, and other secrets in modern computing. Vault handles leasing, key revocation, key rolling, and auditing. Through a unified API, users can access an encrypted Key/Value store and network encryption-as-a-service, or generate AWS IAM/STS credentials, SQL/NoSQL databases, X.509 certificates, SSH credentials, and more.

https://teampass.net/
eamPass is a Passwords Manager dedicated for managing passwords in a collaborative way by sharing them among team members.
Teampass offers a large set of features permitting to manage your passwords and related data in an organized way in respect to the access rights defined for each users.
Teampass is an Open-Source free to use product distributed in respect with OpenSource GNU GPL-3.0.

SAS

Info about SAS-datasets in the WORK-library

April 25, 2018 Root Leave a comment

In SAS Enterprise Guide it is not very easy to see the size and number of observations in datasets in the WORK-library.

The macro below looks in the DICTIONARY.TABLES and gets these info for the WORK-library. Be aware that it will not work for views, because it’s not doing and actual count of the SAS-datasets.

/********************************************************************************
Author        : 
Creation date : 
Description   : Gets info about datasets in the WORK-library.
Example       : %countwork(print);
*********************************************************************************
Input
-----
&print        : If not empty it will do a PROC PRINT of the dataset WORKDS created
                by the macro.
*********************************************************************************
Output
------
WORKDS        : Contains information about the datasets in the WORK-library.
********************************************************************************/
%macro CountWork(print);
          proc sql;
                   create table workds as
                             select    libname
                                                , memname
                                                , typemem
                                                , nobs format=commax10.0
                                                , filesize format=sizekmg.
                                                , nvar
                             from dictionary.tables
                             where libname eq 'WORK'
                             order by nobs
                   ;
          quit;
 
          %if &print. ne %then
          %do;
                   proc print data=workds;
                   run;
          %end;
%mend;

/********************************************************************************

Author :

Creation date :

Description : Gets info about datasets in the WORK-library.

Example : %countwork(print);

*********************************************************************************

Input

-----

&print : If not empty it will do a PROC PRINT of the dataset WORKDS created

by the macro.

*********************************************************************************

Output

------

WORKDS : Contains information about the datasets in the WORK-library.

********************************************************************************/

%macro CountWork(print);

proc sql;

create table workds as

select libname

, memname

, typemem

, nobs format=commax10.0

, filesize format=sizekmg.

, nvar

from dictionary.tables

where libname eq 'WORK'

order by nobs

;

quit;

%if &print. ne %then

%do;

proc print data=workds;

run;

%end;

%mend;

SAS

Microsoft Team Foundation Server (TFS) and SAS

April 9, 2018 Root Leave a comment

Below is shown how you can extract the current Microsoft Team Foundation Server (TFS) revision number for a given file into SAS and use it in your SAS program.

Be aware that Visual Studio 13 is used in the example below. It is uncertain if newer versions of Visual Studio will work.

%macro TFSRev(filename);
    %let tfsver1=;
    %let tfsver2=;
    %let tfsver3=;
 
    /* The default path for the TF program tf.exe with Visual Studio 13. */
    %let TFRev=C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\tf.exe;
 
    filename tfver pipe "call ""&TFRev"" hist ""&filename"" ";
 
    data _null_;
        infile tfver;
        input;
        /* The output is in three different lines. */
        put _n_ _infile_;
        /* Puts the three different output lines in three diffent macrovariables called tfsver1, tfsver2 and tfsver3. */
        call symput('tfsver' !! put(_n_,1.), _infile_);
    run;
 
    /* Suppress output from SAS. */
    options nosource;
    %put Extracting TFS information from: &filename;
    %put;
    %put &tfsver1;
    %put &tfsver2;
    %put &tfsver3;
    option source;
%mend;

%TFSRev(C:\TEMP\sasprogram.sas);

%macro TFSRev(filename);

%let tfsver1=;

%let tfsver2=;

%let tfsver3=;

/* The default path for the TF program tf.exe with Visual Studio 13. */

%let TFRev=C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\tf.exe;

filename tfver pipe "call ""&TFRev"" hist ""&filename"" ";

data _null_;

infile tfver;

input;

/* The output is in three different lines. */

put _n_ _infile_;

/* Puts the three different output lines in three diffent macrovariables called tfsver1, tfsver2 and tfsver3. */

call symput('tfsver' !! put(_n_,1.), _infile_);

run;

/* Suppress output from SAS. */

options nosource;

%put Extracting TFS information from: &filename;

%put;

%put &tfsver1;

%put &tfsver2;

%put &tfsver3;

option source;

%mend;

%TFSRev(C:\TEMP\sasprogram.sas);

You can now use the three diffent macrovariables called tfsver1, tfsver2 and tfsver3 in your program.

Microsoft Visual Studio Team Foundation Server 2013 Power Tools might be needed.
https://marketplace.visualstudio.com/items?itemName=TFSPowerToolsTeam.MicrosoftVisualStudioTeamFoundationServer2013Power

You might also want to have a look at this blogpost regarding SVN.

SAS

Automatically analyzing and documenting SAS-code

March 28, 2018 Root Leave a comment

In SAS Enterprise Guide and SAS-DI you have the possibility to analyze the code for a SAS-program.

The picture below shows the Analyze Program option in SAS Enterprise Guide.

Analyzing the code should result in a conversion of the SAS-code to a SAS Enterprise Guide flow or SAS-DI flow. But none of these code analyzers are very good. Depending on the complexity of the SAS-code you put into the analyzer, they will leave you with a more or less successful conversion. And more times than not they will fail at doing the job.

But SAS comes with a procedure PROC SCAPROC that does a really good job at analyzing and documenting SAS-code.

Below is an example.

proc scaproc;
record '<PATH>' attr expandmacros;
run;
 
proc scaproc;
write;
run;

proc scaproc;

record '<PATH>' attr expandmacros;

run;

proc scaproc;

write;

run;

The links below gives you further descriptions of PROC SCAPROC and its options. There’s also a guide on how to do a graphical presentation of the result fra PROC SCAPROC.

Overview of the SCAPROC Procedure
http://support.sas.com/documentation/cdl/en/proc/61895/HTML/default/viewer.htm#a003199745.htm

Program for parsing the output from PROC SCAPROC to create a data set with inputs and outputs.
http://support.sas.com/kb/58/047.html

Innovative Performance Improvements Through Automated Flowcharts In SAS
http://support.sas.com/resources/papers/proceedings16/11580-2016.pdf

Automatically create diagrams showing the structure and performance of your SAS code
http://support.sas.com/resources/papers/proceedings17/1104-2017.pdf

SAS

Uniqueness in data

March 8, 2018 Root Leave a comment

The SAS-macro below will tell you if a variable in a dataset is unique.

/********************************************************************************
Author        : 
Creation date : ddmmmyyy
Description   : Gets info about uniqness in a SAS-dataset.
Example       : %uniq(sashelp.class, name, print)
*********************************************************************************
Input
-----
&datset   : The dataset to test.
&variable : The variable to test for uniqueness.
&print    : If the output/result should be shown in a PROC PRINT.
*********************************************************************************
Output
------
freq_result          : Dataset sorted with doublets as first rows.
freq_result_doublets : Data containing only the doublets.
********************************************************************************/
%macro uniq(dataset, variable, print);
	proc freq data=&dataset.;
		tables &variable / noprint out=freq_result;
	run;

	proc sort data=freq_result;
		by descending count;
	run;

	data freq_result_doublets;
		set freq_result;
		where count gt 1;
	run;

	proc sql noprint;
		select count(*) into :doublets
		from freq_result_doublets
		where count gt 1
		;
	quit;

	%put --------------------------------------------------------------------------------------------;
	%put NUMBER OF DOUBLETS IN [%upcase(&dataset.)] FOR VARIABLE [%upcase(&variable.)]: &doublets.;
	%put --------------------------------------------------------------------------------------------;

	%if &doublets. eq 0 %then
	%do;
		%put !!!! NO DOUBLETS !!!;
	%put --------------------------------------------------------------------------------------------;
	%end;

        %if &print. ne %then
        %do;
             proc print data=freq_result;
             run;
        %end;
%mend;

/********************************************************************************

Author :

Creation date : ddmmmyyy

Description : Gets info about uniqness in a SAS-dataset.

Example : %uniq(sashelp.class, name, print)

*********************************************************************************

Input

-----

&datset : The dataset to test.

&variable : The variable to test for uniqueness.

&print : If the output/result should be shown in a PROC PRINT.

*********************************************************************************

Output

------

freq_result : Dataset sorted with doublets as first rows.

freq_result_doublets : Data containing only the doublets.

********************************************************************************/

%macro uniq(dataset, variable, print);

proc freq data=&dataset.;

tables &variable / noprint out=freq_result;

run;

proc sort data=freq_result;

by descending count;

run;

data freq_result_doublets;

set freq_result;

where count gt 1;

run;

proc sql noprint;

select count(*) into :doublets

from freq_result_doublets

where count gt 1

;

quit;

%put --------------------------------------------------------------------------------------------;

%put NUMBER OF DOUBLETS IN [%upcase(&dataset.)] FOR VARIABLE [%upcase(&variable.)]: &doublets.;

%put --------------------------------------------------------------------------------------------;

%if &doublets. eq 0 %then

%do;

%put !!!! NO DOUBLETS !!!;

%put --------------------------------------------------------------------------------------------;

%end;

%if &print. ne %then

%do;

proc print data=freq_result;

run;

%end;

%mend;

SAS

Directory for SAS-macros

March 7, 2018 Root Leave a comment

The code below will give you the path of all the directories containing macros in the SASAUTOS in your SAS-session.

%put %sysfunc(getoption(sasautos));
filename _all_ list;

%put %sysfunc(getoption(sasautos));

filename _all_ list;

Windows, Windows 8

Testing a port with PowerShell

January 21, 2018 Root Leave a comment

In newer versions of Windows you are not able to use the Telnet-command to test a port. Don’t worry. In PowerShell v. 4 and onwards you can use the command below.

Test-NetConnection <host> -port <port>

1	Test-NetConnection <host> -port <port>

Eg.

Test-NetConnection www.ryslander.com -port 80

1	Test-NetConnection www.ryslander.com -port 80

DB2, SSIS

SSIS and IBM DB2 – Don’t do it!

July 1, 2017 Root Leave a comment

SSIS is a product that should be indifferent to whatever RDBMS you use. That’s also true. It will work with every RDBMS through drivers ODBC, OLE DB, ADO.NET etc. BUT it works best with Microsofts SQL-server.

This blogpost will take a look at using IBM DB2 with SSIS. Below is a test using ODBC-driver for IBM DB2 from IBM, OLE DB for IBM DB2 from Microsoft and an ADO.NET-driver.

Transfer rate of data

Transferring 100.000 rows from one IBM DB2-server to another IBM DB2-server.

OLE DB-driver for IBM DB2 from Microsoft	6 minuts
ADO.NET-driver	30 seconds
ODBC-driver for IBM DB2 from IBM	5 seconds

As seen above the ODBC-driver for IBM DB2 from IBM is the best solution when it comes to transfer rate. The OLE DB-driver for IBM DB2 from Microsoft is a really poor choice. But using the ODBC-driver for DB2 from IBM is not the solution to everything!

Lookup-task in SSIS

As seen in the picture below it’s only possible to use an OLE DB-connection in the Lookup-task. The Lookup-task is really slow and it might be better doing it in SQL. Regarding a lot of the tasks in SSIS, they are really slow (see links below).

OLE DB Command

SSIS also contains a ‘OLE DB Command’-task and no ‘ODBC Command’-task.

Fast load/Bulk load

OLE DB
It’s NOT possible to use the fast load option with the OLE DB-driver when it comes to IBM DB2. It’s possible to choose the fastload option, but you will get the error below.

The reason why the OLE DB-driver is that slow is probably that it transfers one row at a time.

ODBC
It’s possible to use a batch/fast load load option in ODBC.

ADO.NET
It’s also possible to use a bulk/fast load option in ADO.NET.

Links

The links below is to different sources that can help you with SSIS.

SQL Server Integration Services Design Patterns
A book that describes different design pattern for SSIS.

Task Factory for SSIS from Pragmatic Works
A collection of optimized SSIS-tasks.

Cozyroc
A collection of optimized SSIS-tasks.

PragmaticWorks free online training videos
Free online training videos about SSIS (search for SSIS).

Pluralsight
Online training videos in SSIS (requires a paid subscription to Pluralsight).

SSIS

SSIS and versioning

June 25, 2017 Root Leave a comment

This post will describe the big challenges that SSIS has when it comes to versioning solutions in SSIS when working as a team on the same SSIS-solution.

These experiences are based on using SubVersion (SVN) and Microsoft Team Foundation Server (TFS) for versioning solutions in SSIS. I have no experience with Git. But I will assume that the same problem occurs.

An example could be that John and Allan is working together on a SSIS-solution. Let’s call it ‘Staging DW’. They start out with local (their own) identical working copy of the SSIS-solution.

John creates a new SSIS-package in the SSIS-solution ‘Staging DW’. Let us call the SSIS-package ‘Stage Company’.
Allan creates a new SSIS-package in the SSIS-solution ‘Staging DW’. Let us call the SSIS-package ‘Stage Sales’.

John updates his working copy of the SSIS-solution ‘Staging DW’ and commits his changes to ‘Staging DW’ containing the SSIS-package ‘Stage Company’. Everything goes well.

Now Allan wants to commit his changes to the SSIS-solution ‘Staging DW’. He updates his working copy of the SSIS-solution ‘Staging DW’ – but he gets a conflict in the file ‘Staging DW.dtproj’ (an XML-file containing information about the SSIS-solution ‘Staging DW’ and its SSIS-packages).

The problem is that John’s new SSIS-package ‘Stage Company’ is a new package in the SSIS-solution ‘Stage DW’ and Allans SSIS-package ‘Stage Sales’ is a new package n the SSIS-solution ’Stage DW’. In the XML-file ‘Staging DW.dtproj’ information about the SSIS-package ‘Stage Company’ created by John occupies the same line as information about Allan new SSIS-package ‘Stage Sales’ in the XML-file ‘Staging DW.dtproj’.

The tools for solving conflicts in SVN and TFS isn’t capable of adding to the file. They can only overwrite.

One solution could be that Allan overwrites his ‘Staging DW.dtproj’ file. This means that his SSIS-solution ‘Staging DW’ looses it’s knowledge of his new SSIS-package ‘Stage Sales’. But he doesn’t lose his SSIS-package ‘Staging Sales’ it will still in the directory of the SSIS-solution ‘Stage DW’. He can now add this SSIS-package ‘Staging Sales’ to the SSIS-solution ‘Staing DW’ now containing John’s new SSIS-package ‘Staging Company’ and commit the SSIS-solution ‘Staging DW’ containing both John’s new SSIS-package ‘Stage Company’ and Allan’s new SSIS-package ‘Stage Sales’ without any problems.

Another solution is good old communication between team members. John could have told Allan that he was going to create the SSIS-package ‘Stage Company’ and that Allan should wait a bit with creating his SSIS-package ‘Stage Sales’ until John had committet the SSIS-solution ‘Stage DW’ containing his SSIS-package ‘Stage Company’. Then Allan could have updated his local copy and got the changes John made. Then Allan could start making his SSIS-package ‘Stage Sales’.

Or they could upfront have made all the SSIS-packages (without content) in the SSIS-solution. They could also choose to make SSIS-solution that was so specific or small that they did have to work together on them.

Ryslander.com

Enterprise Password Managers

Info about SAS-datasets in the WORK-library

Microsoft Team Foundation Server (TFS) and SAS

Automatically analyzing and documenting SAS-code

Uniqueness in data

Directory for SAS-macros

Testing a port with PowerShell

SSIS and IBM DB2 – Don’t do it!

SSIS and versioning

Download Microsoft OLE DB v.5 for IBM DB2

implementation, consolidation and innovation