Tutorial: Uploading files demystified

Tutorial: Uploading files demystified

Uploading files to a web server can be a tricky process because a lot depends on the web server’s features, abilities, and limits. One might think that there’s a “standard” upload method, but unfortunately there’s not. How an Apache server running PHP handles things could be drastically different than a .NET server running Visual Basic or C#. Even within a family of servers sharing a common language, the rules vary from server to server and from upload script to upload script.

You must also be very careful about which files should be allowed to upload to your web server. Since you’re creating a script that anybody with the URL can run — and effectively send data to your server — you may open your server to being hacked unless you utilize caution and care. It’s quite easy with server-side scripts to allow the ability to write over critical files with malicious code, so if you don’t know what you’re doing on the server side, you may want to consider other options. In addition, your server may enforce limits on your upload abilities that are beyond your control. For example, many servers limit uploads to small files (under 2 MB), or they may limit the number of files that can be uploaded. All of these limitations must be considered before you begin implementing a file upload process.

Before you begin

Before you proceed with this tutorial, you should read this manual to gain a clearer understanding of the entire process. This tutorial presents a “quick and dirty” method to upload files, but before you use this in production, you must make some adjustments and secure the script as necessary. You should anticipate that it’ll take some effort to get this right!

Using “network.upload()”

Corona’s network.* library contains an API called network.upload() which is a simple method for uploading files to a server, assuming that your server handles the simple method of uploading files using HTTP PUT. Most existing scripts that accept file upload probably won’t work with network.upload() because they’re looking for an HTTP POST form-based MIME multi-part upload format. Corona’s network.upload() API does not talk to these kinds of scripts, but we’ll discuss this further in a bit. First, let’s look at the Corona side…

The Corona script

The script demystified…

As with all network.* API calls, this operates asynchronously, meaning that it will return to your program immediately and process the upload in the background. However, you need to know the status of the upload, in particular when it completes, which is handled by the event listener function:

Let’s inspect this function in more detail. The first thing you must check is whether a network error occurred. This is returned in the event.isError attribute. In network programming, it’s important to understand that this error may indicate that the server is down, it’s unreachable, or it’s timing out. Effectively, it means that you never successfully communicated with the server.

You can successfully connect to and interact with the web server, asking it to do something that it can’t, but it will report a “success” in regards to the isError attribute. In other words, regardless of whether the server sends you a “right” or “wrong” response, you technically had a successful transaction with it. Thus, the else condition block is where you can inspect and handle the various phases of the upload. For instance, in the "began" phase, you may choose to display a widget.newProgressView(). Then in the "progress" phase, increment that widget’s status based on the amount of bytes transmitted. Finally, the "ended" phase lets you know that the web server has completed the upload process.

All finished, correct? Not so fast! The file may still have failed to upload for various reasons, for example, the file was too large, it was not a valid file name, or some permission issue on the server prevented it from uploading. Thus, you should check the event.status attribute which will hold the HTTP “result” code, such as 201 (success) or 403 (permission denied). Depending on the server, there may be additional information in the event.response attribute that can indicate where the problem resided.

Now, let’s re-inspect the various variables and tables required for the network.upload() API call. These include:

  • The URL of the server script to execute.
  • The HTTP method (in this case we’re going to use “PUT”).
  • Some parameters to configure the API Call, including the timeout, body type, and whether you want progress updates.
  • The filename to upload.
  • The source directory where the file can be found.
  • A MIME type string indicating the file type for the server’s use.
  • A custom header for the PHP script (we’ll discuss this further down).

Because this method has no way of telling the server the name of the file to save, we must create a special header named filename that will contain that name (the file we want to use on the remote server). With this step done, add the headers table to the params table and then call network.upload():

The PHP script

For usage with this tutorial, we’re providing a sample PHP script for download. This script is fairly simple and straightforward, but unless you know PHP, it will look intimidating. Before you attempt to modify it for your own project, there are a few important things to understand:

1. Security

When you write a script online, it’s 100% your responsibility to ensure that it cannot be easily hacked. Scripts that write files to the server are the most vulnerable. You may think that only your app will use it, but once it’s on a website, anybody can execute it, figure out the parameters and the methods, and find holes to exploit. We’ve taken some basic steps to prevent this, but you need to take it the rest of the way. This script lets the caller set the filename which is inherently dangerous since people can put in tricks to create false paths that may let them write files to arbitrary locations. Your script should never run at elevated permissions, and your file system should be read-only wherever there are executable script files or system binary executables. Finally, you should do your best to scrub any provided file name before putting this code into production. The safest thing is to prevent the caller from specifying the filename, but this can create usability issues in regards to keeping the files organized.

2. Know your server’s limits

Many PHP servers, if they allow uploads at all, will have very tight limits on the file size that can be uploaded. The code above has a size limit check, but it only works if the server allows larger files and you want to limit the size. In this sample, we allow up to a 5 MB file, but the server itself may only allow 1 MB. The server may not even reach your script if the file is too large, so it’s your responsibility to control the limits. Many websites live in a shared environment with other websites and you should be a good network citizen!

3. Handling files of the same name

The code above uses a simple (but somewhat flawed) method of adding an increasing sequence number to the end of the string. It stops incrementing the number at 100. The while loop that checks for duplicate file names can’t run forever and after you hit 100 uploads of the same name, it will start overwriting the 100th file. Again, this is not production-ready script and you must adapt it to your needs. Another potential method is to get a list of files, find the one with the highest number, parse the name from the number, and increment it.

4. The upload directory

This script assumes that you’ll create a folder named upload as a child folder of where this script is located, but this may not be the best option for your website. This upload folder must have write permissions for the ID your web server runs at. Generally the web server will not run with an ID or within a group that your account has access to. This means the folder needs to be World writable. As a safety precaution, you probably don’t want the folder to have READ or EXECUTE/LIST privileges for the World user. Finally, your server may need to dump the files in a completely different location in your server tree, so you must figure out that path and adjust the PHP script accordingly.

In summary

As you can see, the act of uploading files can be a complex task. Hopefully this tutorial has shed some light on the process. As always, please leave your questions and comments below. You may also download the PHP script based on this tutorial if you wish to use it as a foundation for your own implementation.


Rob Miracle
[email protected]

Rob is the Developer Relations Manager for Corona Labs. Besides being passionate about helping other developers make great games using Corona, he is also enjoys making games in his spare time. Rob has been coding games since 1979 from personal computers to mainframes. He has over 16 years professional experience in the gaming industry.

22 Comments
  • develephant
    Posted at 12:41h, 25 February

    Talk about going above and beyond the call of duty! Thanks for taking the time to share this Rob.

    Is there any particular reason why network.upload() does not support the POST form-based MIME multi-part upload format?

    Best.

  • Nathan
    Posted at 14:02h, 26 February

    Is there an equivalent available for asp/aspx?

    Thanks,
    Nathan.

    • Rob Miracle
      Posted at 15:30h, 26 February

      I’m not an ASP programmer, so I have no clue where to start there.

      • Nathan
        Posted at 15:53h, 26 February

        OK cool.

        • Johnathan
          Posted at 15:44h, 27 February

          Hey Nathan,

          I would recommend going through the Web API documentation at Microsoft if you have a better understanding of MVC type architectures as it utilizes an MVC structure that accepts web requests. If not – I would recommend looking into WCF web services as they are very versatille and you can create web services that accept practically any type of data, such as custom classes, and allows you to do whatever you’d like with it.

          This is a great starting point for Web Api – which in my opinion is very easy to use and nice! http://www.asp.net/web-api/overview/getting-started-with-aspnet-web-api/tutorial-your-first-web-api

          As for the data being passed from Corona to the web service – it wouldn’t matter if it is coming from Corona or jQuery through a webpage. Web Api or WCF only care about the data it receives and that is always standard in terms of HTTP requests/responses (GET, POST, PUT, DELETE). You can easily create a custom type within a WCF web service that contains properties you pass in as an array of that “type” from a JSON request. There are tons of avenues to explore!

          Hope that helps!

  • Ian
    Posted at 00:22h, 28 February

    Hi Nathan,

    I posted this a few weeks ago – uploading files with ASP.Net – but I used network.request and HTTP “POST”. I’ve been using this as a basis for my most recent project and now, using json.encode and then decoding on the .Net side I have multiple files being uploaded in one go which is working nicely.

    http://forums.coronalabs.com/topic/44199-example-file-upload-using-aspnet-and-a-progress-bar/

    Does anyone know if there’s any reason if this approach is no longer appropriate? Should we be using PUT and network.upload instead…?

    Thanks,

    Ian

  • Kerem
    Posted at 16:20h, 28 February

    Rob thank you very much for this very useful tutorial! Very nicely timed.

  • Pat
    Posted at 16:20h, 08 April

    Thanks for the tutorial. I am however having trouble with the event.phase check firing on iOS 7.1 devices (iphone 5S). The application seems to ignore the if-then-elseif block using event.phase. Other event triggers do fire (event.completed) on both the device and simulator, but the event.phase code only works on the corona simulator and fails to execute on the xcode simulator or ios device. any help is appreciated!

    • Rob Miracle
      Posted at 15:45h, 09 April

      Do you have the progress = true, flag in your params?

  • Pat
    Posted at 17:47h, 09 April

    yes, progress = true in the params. The block functions as expected on the corona simulator with the event.phase updates. I know the script executes to completion as the file I am uploading does appear on the server even from device side its just that any display objects I instantiate in the event.phase conditional on the device get ignored. I have tried to debug on the ios simulator but all print statements in the event.phase block get ignored as well.

    • Rob Miracle
      Posted at 16:31h, 10 April

      How big is the file you’re uploading? Perhaps this should be taken up in the forums for further diagnosis.

      Rob

  • Vonn
    Posted at 00:25h, 26 January

    Good Day

    I’ve done a upload from web a html file to a web php and it is successful

    but when I do the app to web php it won’t work. Im using the same php from the web html and app side.

    • Rob Miracle
      Posted at 15:35h, 26 January

      The tutorial explains that all web scripts are not created equally. Most PHP based upload scripts expect Multi-part MIME encoded data. Our network.upload() doesn’t do this type of upload. It’s all explained in the paragraph above under the: Using “network.upload()” header.

  • unknown
    Posted at 05:07h, 30 July

    Hi i use the same code for the camera function to save an image to a database but my simulator get crashed

  • Josep
    Posted at 03:27h, 27 September

    Hi,

    There is any way to do save using a stream? I want to save from the microphone directly to the cloud a very large audio (suppose a 10 hours continuous audio).

    With the described technique I must have a file before on the mobile device and upload it after. As the microphone generates a not compressed audio format it could be to much large in many cases…

    To load and play large audios we can use audio.loadStream but I need something like a audio.saveStream.

    Thanks!

    • Rob Miracle
      Posted at 15:51h, 27 September

      You might be able to implement something with Corona Enterprise, but Corona SDK doesn’t have anything like that.

      Rob

  • Alex Poon
    Posted at 01:37h, 30 October

    I recently moved the script to another server and found that this script is not working. I have already changed the path but the error message I got is not related to the path. I don’t know what should I do next. Hope someone can help.

    When I call the same script, I got an html error 400 said with the following:

    “The page you are trying to access is restricted due to a security rule.

    If you believe the security rule is affecting the normal operation of your website, contact your host support team and provide detailed instructions how to recreate this error.

    They will be able to assist you with rectifying the problem and adjusting the security configuration if needed.”

    What server setting should I change in order to make this upload script work again?

    • Alex Poon
      Posted at 00:54h, 25 November

      I finally fixed it by updating the .htaccess. See the link below if anyone have the same problem:
      https://wordpress.org/support/topic/security-error-1

      Remarks:
      1. My upload directory is under a wordpress site
      2. The new server I am using is SiteGround

  • Sem
    Posted at 01:42h, 26 February

    How do you send variables?

    • Sem
      Posted at 02:12h, 26 February

      I have seen the light.

      If I add a header like this;

      param.headers[myVar] = “ABC”

      …then in the php it appears in;

      $_SERVER[“HTTP_MYVAR”]

      What confused me was that HTTP_ gets added to the front, don’t know why and don’t really care. 🙂

  • Mike
    Posted at 09:43h, 29 July

    @Rob
    Is there any reason why this would not work on a Windows Server? It works fine when I run it on my local MAMP server but it does not work on my Windows Hosting service. I get the following error:

    [-1001: The request timed out.] Network Error.

  • Rob Miracle
    Posted at 12:26h, 29 July

    I don’t know any reason why it wouldn’t work on a Window’s server as long as it honors HTTP PUT requests.