Friday 27 March 2009

Useful Linux Tools for Webmasters

1. Introduction

Recently I found two useful tools that should prove especially useful to developers and maintainers of Internet Websites. They are:

- gftp
- kompozer

You will find a short introduction below.

2. The Gnome FTP Client

It is a GNU and OPen Source visual FTP client (similar to Filezilla or to the well known CuteFTP under Windows) with a GTK+ GUI which is released under GNU license. It is mainly for Linux and Unix systems, but it can be used also on Mac OS and Windows.

It supports the FTP, FTPS HTTP, HTTPS, SFTP e FSP protocols; it supports server proxy for the FTP and HTTP protocols and FXP which allows to transfer files between two remote servers via FTP.

The GUI interface has two main panels with the left one for the local machine and the right one for the remote machine. You can choose the directories of the local and remote machine and request the transfer of the chosen files either from the local to the remote machine or the other way round. Under these two panel the program shows a message queue where the user can check the progress status of the transfers.

There is also a bookmarks menu that allows to quickly connect to remote sites.

In the past I used to work with the line oriented lftp, but I found that I could do a lot more and faster by using this GUI based tool.

I installed the tool, without any problem, on my OpensuSE 10.2 and OpenSUSE 11.0 systems by using an RPM based package.

3. The Kompozer Web authoring tool

Kompozer is an easy WYSIWYG HTML and CSS editor. It is a complete web authoring system that combines web file management and easy-to-use WYSIWYG web page editing capabilities found in Microsoft FrontPage, Adobe DreamWeaver and other high end programs.

The tool has a GUI interface which displays different views of the same document such as the HTML source, the tag or the external view in different panels. You can easily switch between these views and make changes to an HTML or CSS document in the more convenient way. It ihas also an integrated FTP function.

Kompozer is a stand-alone tool with small size and fast speed based on on Gecko, the layout engine inside Mozilla.

I installed the application without any problems on my OpenSUSE 10.2 and OpenSuSE 11.0 systems by using the RPM package which can be downloaded from the kompozer official site.

4. Conclusion

The two products describes above are good examples of functionally advanced applications that can be easily installed on Linux machine

P.S. The article was originally published on http://www.ezinearticles.com
and on http://www.squidoo.com/linuxTools

Monday 9 March 2009

IBM ISeries Utilities For Data Interchange

1. Introduction
The IBM iSeries computers, still commonly known as AS/400 systems use the successful IBM DB2 database to store and access data which are organised into files with the following characteristics:

  • They are structured in data fields which usually have a fixed length and type
  • They have external data definitions which can be used by application programs
  • Numeric data can be stored in packed format where each digit is stored in a semibyte.
  • Data are coded in EBCDIC
Other systems such as Linux/Unix and Windows use files which are just a sequence of bytes (normally coded in ASCII) and therefore are known as stream files. Often their fields have not a fixed length and are delimited by a special field delimiter character such as a semicolon, a colon or a pipe (field delimited files are often produced by exporting data originally stored in spreadsheets such as Microsoft Excel or databases such as Microsoft Access).
It is often necessary or useful to transfer files between the IBM iSeries (AS/400) and PCs and Linux/Unix systems, but the different file organisations described above often make such transfers complicated and painful.
This article described some approaches to simplify the work.


2. IBM useful commands
The IBM AS/400 uses an integrated file system (IFS) that allows to use on the same server different file organisations such as those used by Linux/Unix or Windows and the native AS/400 ones. The native AS/400 files are stored in libraries or DB2 collections within the QSYS.LIB. Other file systems exist in the QOpenSys (similar to Unix) or the QDLS (used to store documents and files in PC formats) environments.
The IFS allows to use on the same server Linux/Unix based applications together with the native AS/400 applications.
The AS/400 Operating system provides some useful commands to simplify the data interchanges between different file systems as described below:

  • CPYFRMIMPF to copy data from IFS to the AS/400 database system
  • CPYTOIMPF to copy data from the AS/400 database system to IFS
  • CPYFRMSTMF to copy stream files into AS/400 database files
  • CPYTPSTMF to copy AS/400 database files to stream files
  • CPYTOPCD to copy AS/400 database files to PC documents, stored in the QDLS folders
  • CPYFRMPCD to copy PC documents in the QDLS folders into AS/400 database files

The CPYTOPCD and CPYFRMPCD commands are specific for the QDLS system and have not many options whereas the other ones allow to use many options and are more flexible. They look similar, but there are important differences as follows:
CPYFRMSTMF converts text files (stream files that are in text format) to physical files. It has no concept of fields, so it can only write records to program described files (i.e. files that have no fields defined) or source pfs.
CPYFRMIMPF converts text files as well, but it tries to interpret fields in the input file and copy them to the appropriate fields in the output file. You can either import delimited fields (for example, comma separated value (CSV) files, tab-delimited files, pipe delimited files, etc) or you can read input from fixed-position fields (you have to define the record layout in a "field definition file")
An example of the second command is the following:

CPYFRMIMPF FROMSTMF('/Fldr1/File1.CSV') TOFILE(Lib1/FILE3) MBROPT (*REPLACE) RCDDLM(*CRLF) DTAFMT(*FIXED) FLDDFNFILE(Lib1/FILE4)

The example above uses a fixed data format (i.e. not delimited) and uses a field definition file (FILE4) to describe the text file fields as follows:
- This is comment
- DBFieldname startpos endpos nullIndpos
field1 1 12 13
field2 14 24 0
field3 25 55 56
field4 78 89 90
field5 100 109 0
field6 110 119 120
field7 121 221 0
*END

The above would be needed to import the text file data into a DB file with field names of: FIELD1, FIELD2, FIELD3,..., and FIELD7. The *END is required. I think you can leave off the third column if no fields are null capable.



3. Some User Utilities
The commands above are useful and normally perfectly adequate, but they can be complex especially when the data fields to be copied are not in the same sequence or when one wants to extract only some data from the text file.
I was involved in a few system migration exercises where such copies between Unix, PCs and AS/400 systems had to be done frequently and I developed therefore some utilities to simplify these activities.
The utilities have the objective to satisfy following requirements:

  • Support of any separator character used to delimiter the fields
  • Possibility to copy valid data into fields defined as alphanumeric, numeric or packed numeric
  • Possibility to copy data stored in different sequences in the two files. For instance it should be possible to copy fields 1, 3,4 and 6 of the text file into the fields 5, 2, 1 and 4 of the target database file.
  • The utility should be able to store the file fields mappings (such as those described above) to allow the user to simply use the previously entered mapping.

I organised the utilities in a few commands and programs as follows:

a) Command UCPYFTP to control the copy between two files. The command requires to enter following parameters:

Copy FTP file to AS400 DB file (UCPYFTP)


AS400 DB File . . . . . . . . . TOFILE
AS400 Library . . . . . . . . . TOLIB *LIBL
AS400 Member . . . . . . . . . . TOMBR *FIRST
FTP File . . . . . . . . . . . . FROMFILE
FTP Library . . . . . . . . . . FROMLIB
Field Separator £T=TAB] . . . . SEPARATOR ']'
Decimal Point . . . . . . . . . DECPOINT '.'
View Field Mappings (Y/N) . . . VIEWMAP 'N'
Up to Record No. . . . . . . . . UPTOREC 0



End



b) Program UFMA01L to display and maintain existing file mappings.

The display looks as follows:



UFMAP30 MAPPING FILE MAINTENANCE 9/03/09 12:42:54

AS400 Database File: WERCSWKF FTP File: WERCS

Seq AS400 Fld Description Typ Len D From
No. Name FldNo
1 WK0003 WK_CLIENTE A 9 3
2 WK0004 WK_CLI_SAP A 10 4
3 WK0005 WK_RAGSOC A 40 5
4 WK0006 WK_ZIP A 5 9
5 WK0016 WK_ADDRESS1 A 100 17
6 WK0017 WK_ADDRESS2 A 100 19
7 WK0101 WK_INDI A 30 0
8 WK0102 WK_CAP A 5 0
9 WK0103 WK_LOC A 25 0
10 WK0104 WK_PROV A 2 0
F3=Exit F6=Update Mappings F9=Use sequential mapping F11=Process



The screen shows that field number 3 of the FTP file is copied to the first field of the database file called WK0003, field 4 is copied to WK0004 and so on.

The fields for which the number is zero are not copied, but are initialised correctly in the target file (with zeros or blanks).
Notice following points:
  • The file to be copied is called FTP file because normally is sent to the AS/400 by using an FTP transfer.
  • The user must specify the name and library of both the FTP file and the target AS/400 database file.
  • The commands use some defaults for the field separator and decimal point, but these can be changed by the user.
  • When the View Field Mappings parameter is set to 'Y', the user will see previously defined field mappings or enter new mappings.
  • When the user invokes the command for the first time on a new couple of ftp and target files, the program extracts the field definitions of the database file and displays the mapping screen to allow the user to enter the field mappings which are then stored to allow a future re-use.


A similar utility called UCPYTOFTP allows to copy the data from an AS/400 database file to a text file to be downloaded to a Linux/Unix or Windows machine.
The Utilities and their documentation can be downloaded for free from the downloads page of my website http://www.datamission.co.uk/.

The programs are free software that can be redistributed and/or modified it under the terms of the GNU General Public Licence as published by the Free Software Foundation.



P.S. The article was originally published on http://www.ezinearticles.com

Web Services With Cobol Or RPG on IBM ISeries

1. Introduction
Web Services and SOAP (or REST) are rapidly becoming state-of-art architectural approaches to allow the access of multiplatform and multilanguage systems,
Many companies have legacy applications which represent a high investment and Web Services promise to allow their access from the web without extensive modifications.
2. An Introduction to Web Services
Web services consist of a group of standards intended to make it possible for diverse systems to communicate, without requiring a particular type of middleware, programming language or even operating system. They can be based on SOAP, REST or XML-RPC,.
SOAP is an open source message standard, based on an XML dictionary, that uses standard transport protocols widely used over the Internet such as HTML or SMTP. It represents a paradigm shift in the way applications communicate with each other, because it does not require any special middleware to allow diverse applications to communicate. The messages are text based and accessible by any application over the network. They are described by using a special XML based language called WSDL.
REST is a simpler type of Web Service where the user simply accesses a URL to obtain back an XML document without any standard specification.
XML-RPC is a way to send commands to another system by using and XML document.
Web service allow to develop loosely coupled applications, by allowing systems based on different technologies and languages to communicate by using open standard largely based on XML.

3. Consuming Web Services from Cobol or RPG
If you need to access a Web Service from a Cobol or RPG program, you can use the IBM Web Services Client for C++ toolkit.
It uses extenal data structures to define the mappings between elements of an XML document and fields usable in Cobol or RPG applications and allows to generate C or C++ functions which can be called by the Cobol or RPG program to invoke the web service. The generation is based on the WSDL file which describes the Web service.
4, Exposing Legacy programs in Cobol or RPG as Web Services

If you want to access legacy Cobol or RPG programs as Web Services, you can use the IBM Toolbox for Java or JTOpen to develop a Java Web Service which calls the Cobol or RPG program and return the results as a SOAP message.
It allows to develop clients in Java or other languages to call legacy Cobol or RPG program and return the results as XML messages according to the Web services specifications.
5. Using PHP with Web Services
PHP has recently been ported to the IBM iSeries systems.
PHP has a good support for Web Services and SOAP especially in the PHP 5 version.
Notice the PHP services can call also call programs written in other languages such as Cobol or RPG and therefore PHP can be an alternative to the approach of the previous point.
6. DB2 Web Services
Data in relational database can be accessed though end-user queries in SQL or other interfaces or by applications that use programming interfaces such as JDBC and SQLJ.
DB2 allows also to use Web services to query the database and to obtain the results as standard SOAP messages.
7. References.

You will find plenty of useful information and tutorials on the above techniques and tools at the URL http://www.ibm.com/developerworks

P.S The article has been originally on http://www.ezinearticles.com