Archive | Writing Code RSS feed for this section

Using OS X APIs directly from Delphi

I spent a good chunk of time last week working on making Usertility work on operating systems other than Windows. The target last week was OS X. In this post, I’ll share a couple of things I learned about working with OS X in Delphi.

Specifically, I’ll talk about using objects directly from the operating system, in order to get access to features that FireMonkey doesn’t directly provide.

Note: This is obviously not cross-platform, since we’re going to be creating platform-specific code. If you’re going to use these techniques in a cross-platform application, you’ll need to rewrite this code for each platform, using $IFDEF to separate code for different platforms. In the case of OS X, you’ll want to enclose the platform-dependent code in {$IF DEFINED(MACOS)}..{$IFEND} pairs.

Rule #1 – The object you have is not the object you want

[Most of] The OS X API is object-oriented, which is cool. The platform API calls often require you to pass an object reference, known as an id in Objective-C parlance. Delphi is also object-oriented, and you can get those OS X objects as Delphi objects pretty easily (I’ll show you how a little below). But here’s the trick: The Delphi object is not the Objective-C id. That will make a little more sense below, but just remember that. If you have an object in Delphi, and you need to pass it to an OS X API, you need to get the id of the object.

With that out of the way, let’s look at doing something common direct from the API: Making an HTTP request.

Our quest: An HTTP request without using Indy

The Indy networking components are included with Delphi XE6, and work in applications created with the FireMonkey framework. Indy is kind of polarizing through, and while many developers use it and are very successful, others prefer not to use it. I’m not making any judgement of Indy, or anyone’s preferences, but for this demo, let’s assume that we want to make an HTTP request without any dependencies other than the operating system.

Step 1: How do we make an HTTP request in OS X?

The first step is figuring out what the OS X APIs would be in the first place. A little web searching leads us to the API docs for NSURLConnection, a class specifically designed for this task.

Looking at the API, we see this list of “tasks:”

NSURLConnection tasks

This is what we Delphi programmers would call the “methods” of the NSURLConnection class. The ones that are prefixed with a ‘-‘ symbol are the “instance” methods, and the ones prefixed with a “+” symbol are the “class” methods.

To keep this simple, we’ll send send a synchronous request, which means that we’re going to send our HTTP request and then wait for a response. (You shouldn’t really do this in your application’s main thread, because it will block your UI. For real-world use, either use a synchronous request in a background thread, or use an asynchronous request). NSURLConnection has a “method” called + sendSynchronousRequest:returningResponse:error:

Let’s look at the documentation for that method:

sendSynchronousRequestThis gives us a little more information:

  • It returns an NSData object
  • It takes 3 parameters:
    • an NSURLRequest, which is in object that represents the URL to load
    • an reference to an NSURLResponse variable, which the method will use as an OUT parameter
    • a reference to an NSError variabls, which the method will use as an OUT parameter. This can be NULL (or nil in Delphi terms)

How do we call this method in Delphi?

To call this class method, we need two things: First, we need a reference to the NSURLConnection class, and second, we need the parameters.

So, here’s the next big idea:

Getting an OS X class reference in Delphi

Many of the OS X classes, including NSURLConnection, are defined in the Macapi.Foundation unit. Add Macapi.Foundation to your uses clause.

If we search Macapi.Foundation.pas for NSURLConnection, we find three apparently useful types:

  • NSURLConnectionClass
  • NSURLConnection
  • TNSURLConnection

We know that we want the sendSynchronousRequest method, and that appears to be defined in NSURLConnectionClass, but NSURLConnectionClass is an interface, and we can’t call class methods on an interface, so how do we get a reference to that interface?

The key here is TNSURLConnection. It’s the “bridge” between the Delphi interfaces that represent the class, and the underlying OS X classes. We can use TNSURLConnection.OCClass to get a reference to NSURLConnectionCass.

IMPORTANT: This is a pattern throughout Delphi. If you want to call a class method on an ObjectiveC class, use T<WhateverTheClassNameIs>.OCClass.callTheMethod(...) (I’m pretty sure OCClass stands for Objective-C Class).

So, we can call TNSURLConnection.OCClass.sendSynchronousRequest(???). We just need to fill in the ???

The first parameter is an NSURLRequest. How do we get one of those?

Getting an OS X Object instance in Delphi

Most Objective-C classes don’t use a traditional constructor the way we’re used to in Delphi. Instead, they often have class methods that return an instance of the class. (If they do use a traditional constructor, it’s actually a pair of methods called alloc and init).

NSURLRequest has a class method +(id)requestWithURL: that returns an NSURLRequest. We’ll use that as our constructor, using the same pattern as above:

//var URLRequest: NSURLRequest
URLRequest := TNSURLRequest.OCClass.requestWithURL(???); //We'll figure out the parameter below

This looks good, but in fact, it’s wrong!

Remember rule #1: The object you want isn’t the object you have. The requestWithURL method returns the id of the Objective-C class. We can’t call Delphi methods on this id. We need the Delphi interface that wraps around this id. Again, TNSURLRequest is the bridge. We can call the wrap method of TNSURLRequest to get the Delphi wrapper interface for the Objective-C id:

URLRequest := TNSURLRequest.Wrap(TNSURLRequest.OCClass.requestWithURL(??)); //This works

IMPORTANT:  This is the other side of the pattern. To get a Delphi interface for an Objective-C class, use T<WhateverTheClassNameIs>.Wrap(ObjC_id);

Of course, now the rabbit hole goes deeper. We need that parameter to requestWithURL, which is an NSURL.

(At this point, we start to get a little frustrated: I just want to give it a string with a URL and get back the data at that URL! Never fear; we’re almost there.)

Looking at the API docs for NSURL, we find a glimmer of hope: NSURL has a class method +(id)URLWithString:. Hopefully that will let us create a url from our string and finally put this all together to get a result.

And indeed, we can. First attempt:

//var URL: NSURL
URL := TNSUrl.Wrap(TNSURL.OCClass.URLWithString('http://www.twodesk.com'));

This almost works, except it doesn’t compile. There’s an error “There is no overloaded version of URLWithString that can be called with these arguments.”

Remember rule #1 again. URLWithString takes an NSString, and we’re trying to pass it a Delphi string. The object we have is not the object we want. Thankfully, there’s a simple utility method to convert a Delphi string into an NSString object. It’s in the Macapi.Helpers unit, and it’s called StrToNSStr:

URL := TNSURL.Wrap(TNSURL.OCClass.URLWithString(StrToNSStr('http://www.twodesk.com')));

Finally! We have a URL object. Let’s put it all together:

var
  URL: NSURL;
  URLRequest: NSURLRequest;
  Data: NSData;
  Response: Pointer;
begin
  URL := TNSURL.Wrap(TNSURL.OCClass.URLWithString(
    StrToNSStr('http://www.twodesk.com')));
  URLRequest := TNSURLRequest.Wrap(TNSURLRequest.OCClass.requestWithURL(URL));
  Data := TNSURLConnection.OCClass.sendSynchronousRequest(URLRequest, @Response,
      nil);
  //Do something with the data
end;

The last step is to do something with the data:

Using NSData

NSData is a general-purpose class for representing any generic data. Its two most important members are NSData.bytes, which is a pointer to the actual data, and NSData.length, which is the size of the data in bytes.

In the case of an NSData object returned by TNSURLConnection in the example above, bytes is UTF-8 encoded text. This means we can directly access the bytes as a PAnsiChar in Delphi:

//var S: string
S := PAnsiChar(Data.bytes);

Put it all together

I created a FireMonkey HD Desktop app, and put a TButton (Button1) and TMemo (Memo1) on the form. Then, I changed the target platform to OS X (If you don’t change the target platform, none of the platform-specific code will compile). Here’s the code for the Button1 click event:

procedure TForm1.Button1Click(Sender: TObject);
var
  URL: NSURL;
  URLRequest: NSURLRequest;
  Data: NSData;
  Response: Pointer;
begin
  URL := TNSURL.Wrap(TNSURL.OCClass.URLWithString(
    StrToNSStr('http://www.twodesk.com')));
  URLRequest := TNSURLRequest.Wrap(TNSURLRequest.OCClass.requestWithURL(URL));
  Data := TNSURLConnection.OCClass.sendSynchronousRequest(URLRequest, @Response,
      nil);
  Memo1.Lines.Text := PAnsiChar(Data.bytes);
end;

Recap

To get direct access to the static methods of an Objective-C class in Delphi: T<ObjCClassName>.OCClass.<StaticMethodName>

To get a Delphi interface to an instance of an Objective-C class: T<ObjCClassName>.Wrap(<ObjC_id>)

(We didn’t use this in the code above, but for the sake of completeness) To get the Objective-C id from a Delphi interface: (<DelphiInterface as ILocalObject).GetObjectID

To convert a Delphi string into an NSStringObject: StrToNSStr(<DelphiString>) //In the Macapi.Helpers unit

Read full story · Comments { 0 }

Bug Dissection: Comparing Filenames

I think it’s good to confess the errors you made when you created bugs in your software. It makes you a better programmer to write out the process of creating and fixing the bug, and might help someone else at the same time. This is one of those confessions.

Castalia often has a need to determine whether two filenames are the same. For example, the code that maintains internal data about the code you’re working on very frequently checks to see if you’re still working on the same file you were last time. It does this by comparing the name of the file you’re working on with the name of the file you were working on last time it checked. If they’re the same, it’s the same file. If they’re different, then you’re working on a different file.

Easy enough, right? Filenames are just strings, so it’s a simple string comparison.

Of course, there has to be a complication. The complication is that the Delphi ToolsAPI doesn’t always give you the filename in the same format. It’s usually the full path to the file, but occasionally, it will be just the extracted filename.

For example, when Castalia gets the name of the current file from the ToolsAPI, it will probably get a string like ‘c:\users\jacob\documents\project1\MainForm.pas’, but it might just get ‘MainForm.pas.’

So, I wrote an IsSameFile routine that would take two filenames as strings, and compare them in both cases (full path, or just filename), to determine if they are, in fact, the same file:

function IsSameFile(N1, N2: string): Boolean;
begin
  Result := CompareText(N1, N2) = 0; //Usually this works, for full paths
  if not Result then //If it's false, maybe we have a simple filename. Check that
    Result := CompareText(ExtractFileName(N1), ExtractFileName(N2)) = 0;
end;

This worked great, until someone came across an edge case that proved to be a bug in this function.

See if you can spot it. I’ll wait…

OK, did you find it?

Here’s the problem: What if the “short” filenames are the same, but are, in fact, different files? For example:

N1 is ‘c:\users\jacob\documents\project1\MainForm.pas’

N2 is ‘d:\projects\project2\MainForm.pas’

The function above will first try to compare N1 and N2, and then when that’s false, will compare the “short” file names, which happen to match, even though it’s painfully obvious that these are not, in fact, the same file.

There are two fundamental errors in logic here. One is thinking there only two cases: 2 full paths, or 2 “short” names. The second is thinking that the second case is always triggered by the first case being false.

The fixes:

First, realizing that there are actually three distinct cases for comparing the two filenames:

  1. Compare two filenames with full paths
  2. Compare two “short” filenames, with no path
  3. Compare one “short” filename with one full path

Second, these cases need to be detected independently. Just because the first case is false doesn’t mean that wasn’t the right case. There needs to be a mechanism to determine which case to use, and then compare the appropriate strings.

Here’s the fixed IsSameFile function:

function IsSameFile(N1, N2: string): Boolean;
var
  S1, S2: string; //Short filenames
begin
  S1 := ExtractFileName(N1);
  S2 := ExtractFileName(N2);

  if (S1 <> N1) and (S2 <> N2) then
  begin
    //2 full path
    Result := CompareText(N1, N2) = 0;
  end else
  if (S1 = N1) and (S2 = N2) then
  begin
    //2 filename only
    Result := CompareText(N1, N2) = 0;
  end else
  begin
    //1 short, 1 path
    Result := CompareText(S1, S2) = 0;
  end;
end;

This handles all three cases correctly, and more importantly, determines which case to use the right way.

This addressed a bug where Castalia’s navigation toolbar would get confused if you opened two files with the same filename, either at the same time, or in sequence (for example, working on MainForm.pas in one project, then closing that project and immediately opening MainForm.pas from another project). That bug is fixed, thanks to the fixed IsSameFile() function, as of Castalia 2014.5.

Read full story · Comments { 14 }

Castalia 2014.2 is now available

I’ve just released a new version of my smart code editor for Delphi programmers: Castalia 2014.2.

Castalia 2014.2 is a bugfix release that addresses two issues:

  • Fixed: “Synchronize Prototypes” doesn’t handle constructors or destructors well
  • Fixed: Syntax error highlighting can get “behind” by a keystroke under some circumstances

Users with a current maintenance subscription can download Castalia 2014.2 at the customer download site.

Everyone else can try Castalia for free at twodesk.com/castalia/freetrial.html.

Read full story · Comments { 0 }

Castalia 2014.1.2 is now available

I’ve been on a bug-fixing rampage, making improvements in Castalia, my collection of awesome tools for Delphi programmers, and it’s time you got to enjoy the benefits.

Castalia 2014.1.2 is available now, with the following bugfixes:

  • Fixed: Smart keys not working in Delphi 2005, 2006, and 2007
  • Fixed: Synchronize prototypes incorrectly handles some variables with ‘const,’ ‘var,’ etc…
  • Fixed: Smart Home key doesn’t work on blank lines
  • Fixed: Synchronize prototypes can leave extra text in the file if formatting conventions weren’t followed closely enough
  • Fixed: Delphi 7 crash on close
  • Fixed: Error CCI 2738 when editing a Delphi unit with no project open
  • Fixed: Castalia doesn’t support [Unsafe] attributes class fields
  • Fixed: Synchronize prototypes doesn’t handle empty parameter lists (empty parenthesis) correctly

Castalia users with a current subscription can download Castalia 2014.1.2 from the download site right now.

Everyone else can try check it out FREE at twodesk.com/castalia.

Enjoy!

Read full story · Comments { 0 }

Dynamic arrays: how they work and how to use them

Let’s talk for a bit about how dynamic arrays work “behind the scenes.” First, a dynamic array primer. If you’re already comfortable with the use of dynamic arrays, you can skip this section:

In the old days, when you needed an array in your Delphi program, the array had to be static, which meant its size had to be known at the time you wrote your code. Those look like this:

var
  Nums: array[0..8] of Integer;

Way back in Delphi 4, dynamic arrays were added to the language. Dynamic arrays can vary in size.  You don’t need to declare its size when you write your code, and you can programatically change the size of the array. Dynamic arrays look like this:

var
  Nums: array of Integer;

This creates a reference to the array, but doesn’t actually allocate any memory for the contents of the array. Before we can put something in the array, we need to set its length using the conveniently named SetLength procedure:

SetLength(Nums, 8);

Now that the length is set, we can assign values to the array, just like we could with a static array:

Nums[0] := 1;
Nums[1] := 2;
Nums[2] := 4;
{...}
Nums[7] := 128;

(Note that dynamic arrays are zero-indexed, so the highest index in the array is Length – 1)

What happens if, during the course of our program running, the size of the array needs to change? With static arrays, this simply wasn’t possible. With dynamic arrays, it’s easy! Just call SetLength again, then you can use the newly allocated elements in your array:

SetLength(Nums, 16);
Nums[8] := 256;
{...}
Nums[15] := 32768;

Even after you’ve changed the size, the original elements are still there – no data is lost (OK, data *IS* lost if you resize an array to be SMALLER than it was before), and you just continue on your way.

So, how does all this magic that makes java developers jealous happen under the hood?

Static arrays are really just pointers to a contiguous area of memory that was allocated to hold the right number of elements for the array. The compiler does the math to turn an array index into a pointer to the into a pointer to a single element in the array.

Dynamic arrays, on the other hand, add a level of indirection. The dynamic array is really a record that holds a few pieces of information about the array, including its size (number of elements), and a pointer to a static array! When you call SetLength, several things happen:

  1. The size field is updated
  2. A new (internal) static array is allocated with the new size
  3. The elements of the old static array are copied to the new array
  4. The static array pointer is changed to the new array
  5. The old static array’s memory is disposed

OK, there’s technically a bit more to it than that. Two points:

  • If there is enough room at the internal static array’s current position to reallocate, AND there is only one variable referencing the array, only step #1 happens. It is, however, good practice to assume that all of this happens every time you change the array’s length.
  • If there are are other variables that refer to the same array, the old static array will not be disposed, because the other variables will continue to point to the old data.

Even with those caveats, for the remainder of this discussion, we’ll assume that all of these steps happen with every change of the dynamic array’s length.

To get the best performance out of your Delphi program, it’s really important to understand the implications of step #3. Whenever you change the size of a dynamic array, the array will be copied, which means iterating through the array in memory. The time this takes is proportional to the size of the array: It takes longer to copy an array of 20 elements than an array of 10.

So what are we to do? We shouldn’t shy away from using dynamic arrays – they’re very useful! We just need to understand best practices for using them. Those best practices depend on the use case.

Here are two common use cases, along with some tips for getting the best dynamic array performance out of them:

First, if you’re reusing a dynamic array variable, but don’t need the old contents after resizing, do the following:

  • Don’t maintain more than one reference to the array. In other words, if A and B are dynamic arrays, never write A := B;
  • Before using SetLength to resize the array, call SetLength(arr, 0); This “resets” the array, and since the new array size is 0, doesn’t copy anything
  • After calling SetLength(arr, 0), call SetLength again to set your dynamic array to its new size

The effects of this are negligible for small arrays, but if you’re using large arrays (how large is hardware-dependent), this will yield significant speed increases. In one of my tests, using this technique shaved almost 10 milliseconds off of the speed of an array resize.

The second case is if your array is going to “live” for a long time, growing continuously. This is typical of any kind of aggregation scenario: an array of strings for a log, and array of numbers being collected for later analysis, etc….

  • Choose a “reasonable” initial length for your array. Make your best guess as to how many elements it might need to hold in a typical situation.
  • NEVER increase the length of your array by 1, or 2, or any constant amount. Never do anything that looks like this: SetLength(arr, Length(arr) + 1);
  • When the time comes to reallocate the array, double its size: SetLength(arr, Length(arr) * 2). Research shows that this strikes the ideal balance between time and space for most cases.
  • Consider using a TList. TList isn’t always the most efficient way to handle a “continuous growth” data structure, but its ease of use might make up for it. For very small arrays (less than a couple hundred element), TList will be just as fast as using a dynamic array with the size-doubling technique, but for anything larger, the dynamic array will be faster.

In a nutshell: dynamic arrays are very useful, precisely because they can be resized, but understanding how to best resize them is important for getting the best speed and memory performance out of your applications.


Take your Delphi programming to the next level with Castalia for Delphi, an advanced code editor for Delphi, integrated right into your favorite IDE.

Read full story · Comments { 10 }

For programmers, by a programmer

Hi. My name is Jacob, and I'm the creator of Castalia.

I starting programming in 1986, learning Lightspeed Pascal on a Mac Classic. Today, I'm a professional programmer, teacher, and entrepreneur.

I have a Master's Degree in Computer Science, and I still love Pascal and Delphi.

I believe that writing code is the heart and soul of software development, and I love helping programmers write code more effectively.