System Interface Workshop Notes - October 12-13, 2011

System Interface Workshop Notes - October 12-13, 2011

Presenters:

Gail Smith – ADFG eLandings Program Coordinator
Paul Caldwell – eLandings Analyst Programmer Supervisor
Ammon Bailey – eLandings Analyst Programmer
Amar Das Khalsa – eLandings Analyst Programmer
John Wilcock – eLandings Consultant
Chris Keller – Wostmann and Associates, eLandings Consultant
Jennifer Mondragon – NMFS Program Manager
Larry Talley – NMFS Analyst Programmer

October 12, 2011


Review eLandings, PTI and tLandings Applications
Introductions and opening comments – Gail Smith
The eLandings development team believes processors will be better served if they pull data from eLandings versus importing data into eLandings from a third party application. The eLandings applications have extensive data validation and business rules that are modified as a result of ongoing actions by the Alaska Board of Fish (BOF), International Pacific Halibut Commission (IPHC) and the North Pacific Fisheries Management Council (NPFMC).
The focus of the workshop will be data extraction and import into a third party business application, but other import/export or full import into eLandings data interface systems are allowable, but not supported.  The Department specifically supports the Public Web Services interface, as documented within the eLandings WIKI - System Interface.

Review of February 2010 XML workshop and Data Extraction versa Data Import – Paul Caldwell
Paul reviewed the System Interface Guidelines, which document a framework for third party developer's support expectations.
Paul provided a brief description of the Processor Tender Interface application (PTI) for tLandings data import. He stated that processors that agree to use tLandings are expected to then import that data into eLandings. The data can also be imported into their own business application, as well. He briefly reviewed the eLandings' online resource, defining XML objects and access to system information.
Gail: Described available user support for eLandings end users, as well as developers, and what assistance we can provide, with emphasis on first accessing existing System Interface documentation and other eLandings system resources located at: https://elandings.alaska.gov/confluence/dashboard.action


System Interface Approaches
Gail provided greater detail on approaches to interfacing with the eLandings system.
Processors with integrated scale weight capture for Pollock create landing reports within their business application and do a manual import into the eLandings System. This approach does not disrupt their NMFS approved method of AFA Pollock weight capture. Few tickets are generated during the season, the landings are few species and the business rules are not complex. The American Fisheries Act (AFA) landing reports are not IFQ/IPQ fisheries.
All other landing reports for this seafood company are reported within the eLandings System and the data is extracted into their business operations system.
Processors can develop their own front-end reporting applications and then import the data into the eLandings system. This approach is not recommended for IFQ/IPQ fisheries as immediate validation of all quota permits is a requirement of the program, validation procedures are complex, and the number of landing reports on any day of the season are not great.
As all landings reports and fish tickets are uniquely numbered and this unique numbering is used within the eLandings system as report identifiers, great care must be exercised to maintain eLandings compatible numbering procedures.
The eLandings system industry applications, eLandings web, seaLandings, tLandings are modified on an as needed basis following actions by our governing bodies – BOF, IPHC and NPFMC. Developing front-end data entry application means that the third party developer will always be modifying the application based upon the needs of the resource management agencies.
The audience was reminded that the current NMFS regulations state that all federally permitted processing facilities must use the eLandings system or an approved application for reporting.
Gail summarized the Import and Export capabilities of the eLandings System. The focus of the workshop is a System Interface that extracts the data from the eLandings System, manually or via Web Services.
Non-web service data extraction tools are available within the eLandings web application. The data extract tool is designed for small to mid-sized processors, as the number of records that can be extracted with any request is limited to one thousand.
The Data Extract tool is available from the eLandings web application, Reports Menu.

The eLandings data extract tool allows the end user to access their authorized operation Landing Report or Production Report records in an XML, CSV or Excel output format. The tool also has the ability to create templates to extract reports in a standardized output of your design.
Documentation on the Data Extract Tool and the Database Field Descriptions are located at: https://elandings.alaska.gov/confluence/display/ifdoc/How+to+Extract+Multiple+Landings+or+Production+Reports+from+eLandings+using+the+Report+Extract+Tool
https://elandings.alaska.gov/confluence/display/ifdoc/Report+Extract+Spreadsheet+Columns
End users were reminded that the XML data extract is the most current and up-to-date version of the database field columns as it is updated whenever a change to the schema occurs. These updates do not occur as frequently in the Excel and CVS format output.
Ammon described the columns available in template download. Most database columns made available for export based upon those that are deemed most likely to be of use by industry. A description of each data field is available at: https://elandings.alaska.gov/confluence/display/ifdoc/Report+Extract+Spreadsheet+Columns
Gail displayed the XML output availability. Randy Swain (NPS) and Ammon point out that a simple copy and paste of the XML from the browser could introduce additional data elements, specifically dashes, which corrupts the XML.

If this occurs, the end user should perform a "Save As" and select the .xml format. This is primarily a limitation on the way browsers render Carriage Return and Line Feed combinations.
eLandings Resources – Brief review – Amar Das Khalas
Amar Das provided a brief review of the eLandings WIKI including:

  • eLandings System Overview

  • User's guide for eLandings

  • Glossary

  • Training Materials

  • System Interface Guide

  • Documentation of XML format of all valid eLandings codes.


He strongly recommended that developers review this documentation carefully.
Lastly, Amar Das displayed our version history of schema and system updates documentation.
Paul pointed out that some of the schema version history is not complete because of the huge number of changes made over the summer.
(Following the workshop, the eLandings Development Team determined that they would keep current only the last year of schema version and system update change documentation.)
He also provided a review of our procedures for system announcements and schema change notification. Registered users associated with an established eLandings operation can be notified of schema changes via email. Participants that are third party developers requested another method to be informed of schema changes, because they are not registered users.
(The eLandings Development Team are actively addressing providing additional and timelier notification of schema changes as registered users.)
Amar Das provided a brief review of the Public Web Services documentation, but noted that this will be next day topic.

A notification discussion followed and Amar Das noted that our email list is for eLandings users associated with an elandings operation, not necessarily 3rd party developers. Update notifications are sent out usually the day before implementation. We recognize that these notifications are not available to TPD (third party developers) that are not working closely with processors.
(This issue is being addressed by the eLandings Development Team to allow TPD access to eLandings change notifications. Need to change how email lists are managed, such as modify to an online subscription system.)
Amar Das and Paul again noted that schema change notification is part of this process, as well as web services change notifications.
Recordkeeping and Reporting Requirements – Gail
Understanding how the eLandings System operates and the imbedded business rules, requires an understanding of the recordkeeping and reporting requirements for all three management agencies – IPHC, ADFG and NMFS. Gail reviewed the on-line documentation for reporting requirements.
Training Scenarios – Gail
Gail reviewed the on-line training scenarios available for large variety of fisheries currently using eLandings. She noted that training scenarios will assist developers in a greater understanding of the eLandings system, reporting sequence, validation and business rules, and allocation calculations.
Gail mentioned that the ADFG maintains a confidential dataset of commercial landings from 1966 to present. She reviewed all three agencies on-line Reporting Requirements and noted that each agency has a set of rules pertains to fisheries where their jurisdiction exists. The jurisdiction is related to state/federal waters, specific target fisheries, such as halibut and if the species is state or federal managed. These jurisdiction trigger when a Landing Report or Production Report is required. She specifically reviewed the location of documentation for groundfish and IFQ reporting requirements. Reporting requirements and required data elements are the result of the public management council/board/commission process, and there is often a disconnect in timing of decisions changing requirements and notification to programmers and developers of the changes. These rules and changes drive database/application business rules.
Randy (NPS) pointed out that the word "expectation" is used, but wants to know where the line exists between expectations and requirements. He used, as an example, developing his own program to upload to eLandings instead of PTI.
Gail pointed out that regulations state that fish tickets must be submitted within a specified time period – that is a requirement as we must have timely reporting. The eLandings Team and ADFG expect industry that are using tLandings for electronic reporting on-board tenders to use PTI to upload this landing report data into eLandings because the PTI performs additional data validation and because ADFG wants the data in an electronic format. We do not want processors to use tLandings and then only submit the printed PDF copies of the electronically generated fish tickets to the Department.
The ADFG does not require eLandings or tLandings, but do believe that there are significant benefits to it for processors. The ADFG Commissioner and the Director of Commercial Fisheries have stated that eLandings is the desired direction of the ADFG for reporting of commercial catch. Eric (Coastal Villages) commented that they have used tLandings for two years with minimal issues. They extract the data from eLandings into their accounting systems and can pay Bethel based fishers from their Anchorage office within four days.
She also added that the workshop is designed to provide information on what is required, by regulation and what is expected by the program managers.
eLandings System Overview – Gail
The system has five visible components.