What is HTTP POST in the PostURL?

PostURL is a feature used in FileCatalyst TransferAgent that enables web applications to retrieve the list of files that were uploaded to FileCatalyst server. Instead appending file names to the URL via the HTTP GET method the HTTP POST is used instead. On most browsers HTTP GET is limited to about 2000 characters, thus if hundreds of files are uploaded, the GET URL can be truncated and the server side script will only receive a partial list of uploaded files. The limit of HTTP POST is usually around 10MB which is large enough for even several thousands of file names to be sent to the web application.

PostURL parameter must be always set with ABSOLUTE URL starting with http:// or https://. For example, "doAfter..." can be set to a path relative to the location of the TransferAgent however PostURL must contain the entire absolute path to the script that will process the PostURL.

TransferAgent will continue to transfer files even after the browser/tab is closed. Therefore it's recommended to use the PostURL for any tasks after transfer. Using PostURL will ensure that the list of files is properly delivered to the web application even after the browser/tab is closed.

PostURL and unique sessions in a web application

There is a common problem with web applications. When a user logs into a web application, a unique sessionID is usually created. This ID is usually stored either via a session cookie or if cookies are disabled it can be appended to the URL. To bring the list of files transferred into a user's session, a special page for retrieving postURL information has to be created.

The problem is that the TransferAgent will not have access to the session cookie stored by the browser. Therefore a new session is created by the server every time the TransferAgent does an HTTP POST to a PostURL. Because sharing of information between sessions is usually not permitted, the following workarounds can be used:

1. Using a server side script to write the contents of the HTTP POST into a file then retrieving the contents of that file by the server side script contained in "doAfter..." :- In this method, server side script in "PostURL" will read the parameters passed by the TransferAgent in HTTP POST and write this information to a file. Then another server side script in "doPost" reads the data from the saved file.
   
   
   For example:
   Let's assume that we set the following parameters:
   
   PostURL : "postProcess.php"
   doAfterTransfer : "succeed.php"
   
   The TransferAgent will do a HTTP POST to postProcess.php with the following parameters:
   • "f"- contains a | (pipe) delimited list of the full remote paths that were transferred or schedule to be transferred
   • "lf" - contains a | (pipe) delimited list of the full local paths that were transferred or schedule to be transferred
   • "s" - contains a | (pipe) delimited list of the sizes of each file
   • "status" - Status of each file transferred. "0" = File transfer was not attempted, "1" = File transfer was successful, "2" = File transfer was cancelled, "3" = File transfer failed with an error. For example, if the transfer included 5 files, and all files were transferred successfully, you would see status=11111. If the transfer was cancelled during the 3rd file, you would see status=11200.
   • "allfiles"- contains a | (pipe) delimited list of the full paths that were transferred or schedule to be transferred, this list has a 1 to 1 relationship with the status value where each status value corresponds to each value in allfiles
   
   postProcess.php is configured to retrieve all the file names and store the list of file names in some arbitrary text file (for example, succeeded.txt)
   
   After the files are uploaded and PostURL is called, the TransferAgnet will redirect the browser to a new URL set in doAfterTransfer (succeed.php) where the list of files can be retrieved from succeeded.txt
   succeed.php is configured to open succeed.txt and retrieve the data submitted by the TransferAgent via HTTP POST to postProcess.php. At this point the programmer will have full access to user's session and can do whatever is required by the web application with the list of the file names. Finally succeed.txt can be discarded once succeed.php has processed the list of all the file names.
2. The second option is very similar to the first method but instead of writing the list of file names into a file on the server, the file list is stored in a server data object. For example In JSP/Servlet this can be done via getServletContext().setAttribute("ATTRIBUTE_NAME", {file_list_object}); The {file_list_object} can then be retrieved from any JSP/Servlet with getServletContext().getAttribute("ATTRIBUTE_NAME");
3. A less recommended option is to append the session ID to posturl - The syntax of how to append session ID to posturl varies with every technology. In general the posturl will look as follows:
   PostURL:"http://some-server.com/postProcess.php?session=<SESSION-ID>" This option is not always possible because re-writing of the session ID in the URL is not supported by all the server side technologies. Also some technologies don't allow attaching a sessionID to a URL due to security constraints. For example: Apache-Tomcat 6 allows, re-writing of the session ID in the URL but in Apache-Tomcat 7 this feature is disabled.

Script Examples

Below are some script examples written in the most common web programming languages that can be used as a starting point for the PostURL parameter. The alternative way of re-writing the URL with the SessionID in the PostURL is also provided. The scripts below should be used as an example or simply as an illustration on how to properly implement PostURL.

<?php
/* CODE TO PARSE AN HTTP POST REQUEST AND STORE THE DETAILS IN A FILE */
/* Script to Output File Names Received By Applet Author: Unlimi-Tech Software, Inc. */
$filePostVariable = 'f'; //The POST variable containing the file listing with full paths
$fileDelimiter = '|'; //The delimiter that seperates the file list
$fileOutput = 'succeeded.txt'; //The file location to write the file list
/* Script Start */
//Get the file POST data
$filePost = !empty($_POST[$filePostVariable]) ? $_POST[$filePostVariable] : '';
//Break the file list into an array
$filePost = explode($fileDelimiter, $filePost);
//Initialize the output variable
$outputData = '';
if (is_array($filePost)){
	//Loop through the file list
	foreach ($filePost as $fileName){
		if ($fileName){
			//Append the filename to the output variable
			$outputData .= $fileName . "\r\n";
		}
	}
}
//Create and/or open the file with write permissions
$handle= fopen($fileOutput,'w');
//Flush the output to the file
fputs($handle, trim($outputData));
//Close the file
fclose($handle);
/* End Script */
?>
<html>
<!--TransferAgent ignores the response from the server. There is no need to redirect or output
anything besides the empty HTML tag-->
</html>

<%@ page language="java" import= "java.io.*,java.lang.*,java.util.*"%>
<%

String fileOutput = "succeeded.txt"; //The file location to write the file list
String remotepath = request.getParameter("f"); //Getting the full remote path 
String localpath = request.getParameter("lf"); //Getting the full local path 
String status = request.getParameter("status"); //Getting status of the files 
String size = request.getParameter("s"); //Getting size of the files 
// Delimited list of the full paths that were transferred or schedule to be transferred,
// this list has a 1 to 1 relationship with the status value 
// where each status value corresponds to each value in allfiles
String allfiles = request.getParameter("allfiles");

%>
<%
try {
	File cf = new File(".");
	String sep = System.getProperty("file.separator");

	String absPath = cf.getAbsolutePath();
	System.out.println("absPath: " + absPath);
	absPath = absPath.substring(0, absPath.lastIndexOf('.'));
	String currPath= absPath+fileOutput; //getting the location to store the file 
	BufferedWriter bw = new BufferedWriter(new FileWriter(new File(currPath)));

	//This section will perform a write to the file specified in the fileOutput field 
        //include details about the transfered files.
	bw.write("The list files queued:	"+allfiles); //writing list of all files that are queued to the output file	
	bw.newLine();
	bw.newLine();
	bw.write("Remote Path:  "+remotepath); //write the remote paths attribute to ouput file
	bw.newLine();
	bw.newLine();
	bw.write("Local Path:  "+localpath); //writing only local path to the output file
	bw.newLine();
	bw.newLine();
	bw.write("File Status - "+status); //writing file status to the output file	
	bw.newLine();
	bw.newLine();
	bw.write("File Size - "+size); //writing size to the output file
		bw.close(); //closing the file 
	}
catch (IOException e) {
	throw e;
}
%>

<%@ Page Language="C#" Debug="true" %>
<html>
<head>
<title>Post URL using ASP C#</title>
</head>
<body>
<%
String info = Request.Form["f"];
// define which character is separating fields in this case it is ;
char[] delimiter = {'|'};
String output="";
string[] arInfo = new string[30];
arInfo = info.Split(delimiter);
for(int x = 0; x < arInfo.Length; x++){
    output = output + arInfo[x] + System.Environment.NewLine;
}
System.IO.TextWriter tw = new System.IO.StreamWriter("C:\\succeed.txt");
tw.WriteLine(output);
tw.Close();
%>
</body>
</html>