tag:blogger.com,1999:blog-40370792215000147252024-03-12T20:05:47.103-04:00Everyman SoftwareMusings on software development, open source, PHP and current projects.Josh Adellhttp://www.blogger.com/profile/13146384900735324084noreply@blogger.comBlogger58125tag:blogger.com,1999:blog-4037079221500014725.post-1419360850736498702013-05-01T10:17:00.001-04:002013-05-01T13:29:24.897-04:00Serializing Data Like a PHP SessionPHP has several built-in ways of serializing/unserializing data. The most cross-platform is <code><a href="http://php.net/manual/en/function.json-encode.php">json_encode</a></code>; pretty much every programming stack can JSON decode data that has been encoded by any other stack. There's also PHP's native <code><a href="http://php.net/manual/en/function.serialize.php">serialize</a></code> function, which is not as cross-platform, but has the added benefit of being able to store and restore PHP objects to their original class.<br />
<br />
There's a third, and much lesser known PHP serialization format: the format that PHP uses to store <a href="http://php.net/manual/en/session.examples.basic.php">session data</a>. If you have ever popped open a PHP session file, or stored session data in a database, you may have noticed that this serialization looks very similar to the <code>serialize</code> function's output, but it is not the same.<br />
<br />
Recently, I needed to serialize data so that it looked like PHP session data (don't ask why; I highly suggest not doing this if it can be avoided.) It turns out, PHP has a function that encodes data in this format: <code><a href="http://php.net/manual/en/function.session-encode.php">session_encode</a></code>. Great! I'll just pass my array of data to it and...<br />
<br />
Oh wait. <code>session_encode</code> doesn't accept any arguments. You can't pass data to it. It just takes whatever is in the <code>$_SESSION</code> superglobal and serializes it. There is no built-in function in all of PHP that will serialize arbitrary data for you the same way that it would be serialized into a session.<br />
<br />
There are a few userland implementations of PHP's built-in session serialization, mainly built around string splitting and regexes. All of them handle scalar values; some handle single-level arrays. None of them handle nested arrays and objects, and some have trouble if your data contains certain characters that are used in the encoding.<br />
<br />
So I came up with my own functions for reading/writing arbitrary session data without overwriting the existing session. (<em>Edit: I later noticed that a few people suggest a similar method in the comments on the PHP manual pages for <code>session_encode</code> and <code>session_decode</code>.</em>):<br />
<br />
<script src="https://gist.github.com/jadell/5495413.js"></script><br />
Using these functions requires there to be an active session (<code><a href="http://php.net/manual/en/function.session-start.php">session_start</a></code> must have already been called.) <em>Edit: Thanks to Rasmus Schultz for also pointing out that <code>session_encode</code> might be disabled on some systems due to security concerns.</em><br />
<br />
PHP already has a built-in way to serialize and unserialize session data. The problem is that it only serializes from and unserializes into the PHP $_SESSION global. We probably don't want to overwrite the current $_SESSION. We hold a copy of whatever data is already in $_SESSION, then use it to perform our data serialization, then restore it afterwards. And because we're using PHP's built-in session serialization, we get nested array and object serialization for free, and we didn't have to write our own parser.Josh Adellhttp://www.blogger.com/profile/13146384900735324084noreply@blogger.com6tag:blogger.com,1999:blog-4037079221500014725.post-48292287179220754242013-04-23T09:30:00.000-04:002013-10-22T19:23:41.863-04:00Loggly and Puppet<i>Update 2013-10-22: This post refers to Loggly generation 1, and may (most likely) not work with <a href="http://www.loggly.com/bright-new-day-second-generation-loggly/"> Loggly's new second generation product offering.</a></i><br />
<br />
As a follow-up to my previous post on <a href="http://blog.everymansoftware.com/2013/04/loggly-from-javascript.html">pulling data from Loggly using JQuery</a>, this post will show how to use <a href="https://puppetlabs.com/">Puppet</a> to automatically register and configure instances to send data to <a href="http://www.loggly.com/">Loggly</a>.<br />
<br />
At <a href="http://www.servicetrade.com/">ServiceTrade</a>, we use <a href="http://aws.amazon.com/">Amazon Web Services</a> for almost all of our infrastructure. All our production servers are EC2 instances. The configuration of all the instances is kept in Puppet manifests. Instances go down and come up all the time, and Puppet helps us make sure they are all configured exactly alike out of the box.<br />
<br />
A server cannot send data to Loggly unless you have previously told Loggly to accept data from it. Unfortunately, with server instances being created and removed automatically, it would be impossible to keep up with hand-registering each instance with Loggly. Fortunately, we can use Loggly's API and some Puppet manifests to register our instances for us when they come up.<br />
<br />
We use <a href="http://www.rsyslog.com/">rsyslog</a> on our instances for collecting system log data (syslog, kernel, mail logs, etc.). rsyslog can tail log files forward them to other log files, or even other servers via TCP or UDP. Loggly has great documentation on <a href="http://loggly.com/support/sending-data/logging-from/syslog/rsyslog/">setting up rsyslog to forward log files to Loggly</a>.<br />
<br />
First, we need to have Puppet manage rsyslog. This ensures that rsyslog will be installed, and gives us control over rsyslog's master configuration file and a directory of instance specific configuration files. Below is the rsyslog module file. <em>All files are relative to the Puppet root directory.</em><br />
<br />
modules/rsyslog/manifests/init.pp<br />
<script src="https://gist.github.com/jadell/5440543.js?file=modules-rsyslog-manifests-init.pp"></script><br />
As it says, the main configuration file will be in <code>modules/rsyslog/files/rsyslog.conf</code>. The config file is the standard one installed by our package manager with a few minor alterations, seen here:<br />
<br />
modules/rsyslog/files/rsyslog.conf<br />
<script src="https://gist.github.com/jadell/5440543.js?file=modules-rsyslog-files-rsyslog.conf"></script><br />
That last line is important, because all out Loggly specific configurations will go in <code>/etc/rsyslog.d</code>.<br />
<br />
Now that rsyslog is set up, we need to tell each instance where to send its log files. Additionally, we need to register each instance with each log file it will be sending to Loggly. Each log file is sent to a different endpoint, which Loggly refers to as an input. Each input has an ID, and a specific port on the Loggly logging server that maps to that ID. We have already set up a specific Loggly user for API purposes, and we'll use that user to do the registration.<br />
<br />
First, we set up a new module that will hold our Loggly API configuration.<br />
<br />
modules/loggly/manifests/init.pp<br />
<script src="https://gist.github.com/jadell/5440543.js?file=modules-loggly-manifests-init.pp"></script><br />
The hash of inputs allows us to easily reference each input we've mapped in Loggly, without having to remember specific ID and port numbers elsewhere in our manifests.<br />
<br />
We'll also set up a Puppet template for tailing out log files and forwarding them to Loggly:<br />
<br />
modules/loggly/templates/rsyslog.loggly.conf.erb<br />
<script src="https://gist.github.com/jadell/5440543.js?file=modules-loggly-templates-rsyslog.loggly.conf.erb"></script><br />
The template will take the values defined in loggly::$inputs and create one log file per input, which will ultimately end up in <code>/etc/rsyslog.d</code>.<br />
<br />
One last Loggly manifest file is needed. This one generates the config file for an input, then registers the server against the input using Loggly's API.<br />
<br />
modules/loggly/manifests/device.pp<br />
<script src="https://gist.github.com/jadell/5440543.js?file=modules-loggly-manifests-device.pp"></script><br />
This manifest uses the $name passed to the definition to gather data from the $inputs hash to build a config file. Also, it execs a cUrl call to Loggly's API to register the device for the input. The response to this call is stored for two reasons: first, if anything goes wrong, we have a record of Loggly's response to our request; and second, if the response file already exists, Puppet will know it does not need to make another call to the API.<br />
<br />
All that remains is to use our loggly::device definition in a node definition:<br />
<br />
manifests/site.pp<br />
<script src="https://gist.github.com/jadell/5440543.js?file=manifests-site.pp"></script><br />
Since our input IDs and ports are bound to specific input names in our $inputs hash, we only need to know the names of the inputs we want to configure this instance to send to, and loggly::device does the rest.<br />
<br />
Hopefully, at some point we (or someone else) will get around to releasing a proper Puppet module for this. Until then, I hope this post helps you get set up with the Loggly centralized logging service.Josh Adellhttp://www.blogger.com/profile/13146384900735324084noreply@blogger.com3tag:blogger.com,1999:blog-4037079221500014725.post-20539177025544413642013-04-12T23:47:00.001-04:002013-10-22T19:23:26.766-04:00Loggly from Javascript<i>Update 2013-10-22: This post refers to Loggly generation 1, and may (most likely) not work with <a href="http://www.loggly.com/bright-new-day-second-generation-loggly/"> Loggly's new second generation product offering.</a></i><br />
<br />
For my most recent Dev Days project, I implemented centralized logging for our application, <a href="http://servicetrade.com">ServiceTrade</a>. I don't want to worry about running our own indexing server, or storing the logs long term, so I investigated several SaaS logging solutions and eventually settled on <a href="http://www.loggly.com/">Loggly</a>. I was impressed with the ease of setting up our account, defining our logging inputs and even integrating with our <a href="https://puppetlabs.com/">Puppet</a> configuration management infrastructure. For long term storage, they push raw log files to an S3 bucket of your choosing. Their customer support seemed very eager to help with the one issue I had. All-in-all, I've been pleased with the product.<br />
<br />
One thing about Loggly that could use a little work is saved searches. First off, when Loggly gives you a graph of events from a saved search (a very cool feature) the graph sometimes loses information when zooming in and clicking on a section to see specific logs. Visiting the page for a saved search on a specific set of inputs and clicking on the graph to pull up the log lines for that search with give log lines across <b>all</b> inputs, not just the ones the saved search is limited to.<br />
<br />
<strike>Secondly, you are limited to only 5 saved searches at the moment. The saved search feature is in beta, so hopefully they will allow saving of more (ideally unlimited) searches in the future.</strike> Apparently, you can have up to 2000 saved searches; the wording on the saved searches list page is out-of-date.<br />
<br />
We are using their excellent API to pull down data and do our own visualizations of multiple saved searches. I'm using jQuery on a simple HTML page to query the API. There are a few caveats. The following information will hopefully prevent someone else from spending the half-hour I did trying to figure this out.<br />
<br />
First of all, the API uses HTTP basic authentication. Jquery's <a href="http://api.jquery.com/jQuery.get/">get</a> call does not handle HTTP authentication, so I had to use the more verbose <a href="http://api.jquery.com/jQuery.ajax/">ajax</a> method.<br />
<br />
Also, since the request is cross-domain, I had to use JSONP, which Loggly supports.<br />
<br />
Finally, Loggly's API returns a Bad Request response if you send any parameters that it does not recognize. Unfortunately, unless you tell it otherwise, jQuery.ajax() will always send a timestamp query parameter to prevent response caching. In order to get everything to work, I had to tell the request to turn caching off and not send the parameter.<br />
<br />
Here is what the final call looks like:<br />
<script src="https://gist.github.com/jadell/5376837.js"></script><br />
You might also want to check out my blog post soon on <a href="http://blog.everymansoftware.com/2013/04/loggly-and-puppet.html">using Puppet to automatically register servers with Loggly</a>. Josh Adellhttp://www.blogger.com/profile/13146384900735324084noreply@blogger.com4tag:blogger.com,1999:blog-4037079221500014725.post-80989628987078093252013-02-11T08:30:00.000-05:002013-02-11T08:30:03.135-05:00Storytellers and Prognosticators: Lessons in Communication StyleEvery other Friday, my team holds a retrospective meeting. One issue that came up in our most recent retrospective was a particularly contentious user story estimation meeting that had occurred a few days prior. Voices were raised, people were interrupted mid-sentence, and sarcasm was liberally deployed. This was a specific isolated incident, and I am very proud of my team that we were able to quickly own, discuss and remedy the situation. We are a better team of communicators as a result.<br />
<br />
After the retrospective, I thought about different communication styles, and I tried to come up with different personas for the different types of communicators on my team. So far, I've broken the communication styles down into these general personas:<br />
<br />
<strong>Storytellers</strong> love to talk about the past. Their goal is to remind everyone that experience is the best teacher. They are the archive of a team's experiences, reminding everyone of past obstacles and past triumphs. By trying to couch everything in terms of previously encountered situations, storytellers sometimes have trouble recognizing changed circumstances. You can recognize a storyteller by phrases like "Do you remember when..." and "Last time this happened..."<br />
<br />
<strong>Prognosticators</strong> are in some ways the opposite of storytellers. They love to talk about likely outcomes and visions of the future. Their ideas tend to be expressed as innovative approaches to problems and as warnings about potential pitfalls. Prognosticators can sometimes derail a conversation by making predictions based on erroneous assumptions. They can be recognized by phrases like "Something we need to watch out for..." and "It's possible that..."<br />
<br />
Prognosticators and storytellers tend to feed off each other, with the latter talking about how a current situation is similar to the past, and the former talking about how the current situation is different. When they get on a roll together, it may be difficult for the other personas to break into the conversation. Both must be careful to not take the conversation off tangents and drive it away from productive outcomes.<br />
<br />
<strong>Inquisitors</strong> communicate by asking questions (the Socratic method.) Their strength is getting others to think about what they are saying by asking for clarification, and by steering the tone and direction of the conversation to discover new possibilities. Inquisitors are excellent at keeping storytellers and prognosticators on track. It is important that inquisitors remember to contribute knowledge instead of always asking questions to which they already know the answers, otherwise they risk looking condescending. Inquisitors can be recognized by phrases like "What if..." and "What did you mean by..." and "Have you considered..."<br />
<br />
<strong>Evaluators</strong> are good listeners. Unlike the other personas, which tend to drive conversations, evaluators do not speak often, and when they do, they tend to be soft-spoken. Evaluators are good at judging ideas objectively and combining multiple points of view into one cohesive vision. It is important that evaluators do not let themselves get talked over, and that they do not completely hold back from contributing; inquisitors can help draw an evaluator into the conversation. Evaluators can be recognized by phrases like "That's an interesting thought..." and "I was thinking about..." <br />
<br />
Most people are more comfortable in one, or a combination of two, personas. I call this their <em>base communicator</em>. These are not discrete communication styles. Elements of any one may be combined with elements of another, and no person is purely one persona or another. From day to day, even within the course of a single conversation, people flow in and out of these different personas. <br />
<br />
In conversation, especially in larger groups, it is important to recognize which persona is speaking. Remember that familiarity breeds complacency! As team cohesion grows, you will soon learn to recognize each team member's base communicator, and when that happens it is even more important to pay attention to what they are saying and how they are saying it. Just because you know someone's general communication style, you cannot assume that they will always communicate that way in every circumstance.Josh Adellhttp://www.blogger.com/profile/13146384900735324084noreply@blogger.com1tag:blogger.com,1999:blog-4037079221500014725.post-11817163702988744722012-11-21T09:00:00.000-05:002012-11-21T09:00:10.753-05:00Migrating to Dependency InjectionRecently, I gave a lunch-and-learn to my team on the topic of Dependency Injection (DI). Instead of showing a bunch of slides explaining what DI is and what it's good for, I created a small project and demonstrated the process of migrating a codebase that does not use DI to one that does.<br />
<br />
Each stage of the project is a different tag in the repository. The code can be found on Github: <a href="http://github.com/jadell/laldi">http://github.com/jadell/laldi</a>. Checkout the code, and run <code>composer install</code>. To see the code at each step in the process, run the git checkout command in the header of each section.<br />
<br />
<h3><code>git checkout 0-initial</code></h3><br />
The initial stage of the project is a listing of developers on the team, and with whom each has pair programmed. The project uses <a href="http://silex.sensiolabs.org/">Silex</a>, which is both a PHP micro-framework, and a simple dependency injection container. For the initial stage, only the framework is used (for HTTP request handling, routing and dispatching.) Each route handler is registered in index.php. A handler intantiates a controller object, then calls a method on the controller to handle the request. For this project, there is only one controller, the HelloController, but there could be many others.<br />
<br />
When the HelloController is instantiated, it instantiates a UserRepository to handle access to User domain objects and a View to handle rendering the output. The UserRepository instantiates a Datasource to handle access to the raw data. The path to the data file is hardcoded in the Datasource. User objects instantiate a UserRepository of their own, so they can access the list of paired Users.<br />
<br />
<script src="https://gist.github.com/4122568.js?file=0-initial.php"></script><br />
The code smell here is the chain of initializations that get triggered by constructing a new HelloController, which leads to several problems: it tightly couples every component to every other component; none of the components can be unit-tested in isolation from the others; we can't change the behavior of a class or its dependencies without modifying the class; and if the constructor signature of any component changes, we need to find each instantiation of that component and change it to pass in the new parameters (which might involve threading those parameters through multiple layers which don't require them.) There also may be bugs with multiple objects having their own copies of dependencies (for example, each User object should probably share a UserRepository, not have their own.)<br />
<br />
<h3><code>git checkout 1-add-di</code></h3><br />
A simple way to solve these problems is to ensure that no class is responsible for creating its own dependencies. In order for a class to access its dependencies, we pass those dependencies into the class's constructor (another form of DI uses "setter" injection instead of constructor injection.)<br />
<br />
<script src="https://gist.github.com/4122568.js?file=1-add-di_classes.php"></script><br />
The problem now is that all the dependencies must be wired together. Since no class instantiates its own dependencies, we must have a component that does this for us. This is the job of the Dependency Injection Container (DIC).<br />
<br />
Silex is also used as a DIC. In bootstrap.php, all class instantiation is wrapped in a factory method that is registered with the DIC. When the DIC is asked for a registered object, if that object is not already instantiated, it will be created. Additionally, all its dependencies will be created and injected through its constructor.<br />
<br />
For cases where a component may need to create multiple objects of the same type (like the UserRepository needs to create many User objects), factory closures are created. The factory makes sure that any User object is created with the correct dependencies, and that shared dependencies are not re-created each time. Since the closure is defined in the DIC, the UserRepository does not need access to any of the external dependencies that might be used by User objects, and the closure can be passed to the UserRepository.<br />
<br />
<script src="https://gist.github.com/4122568.js?file=1-add-di_bootstrap.php"></script><br />
<br />
<h3><code>git checkout 2-comments</code></h3><br />
Our project gets a new requirement to allow comments on users. Following the same pattern as with Users, we create a CommentRepository and a Comment domain object, and factory methods for creating and injecting each. HelloController needs access to the CommentRepository in order to save comments. Also, Users need access to the repository to retrieve all the Comments for that User. Since we are using DI, we change their constructors to receive a CommentRepository.<br />
<br />
<script src="https://gist.github.com/4122568.js?file=2-comments.php"></script><br />
If we were not using a DIC, we would have to find every place that instantiated a HelloController or User object and change those calls to pass in the CommentRepository. Additionally, we would have to pass the CommentRepository through to any place that instantiated the object. Since we are using a DIC and factory methods, we can be assured that there is only one place that creates each of these object types, and therefore only one place needs to change (the factory methods in the DIC.)<br />
<br />
We already get benefits from our refactoring to use a DIC!<br />
<br />
<h3><code>git checkout 3-events</code></h3><br />
Our last requirements are to do some simple tracking of page views and notification of comments (perhaps with email, but for demonstration purposes, to a log file.) Each of these is a good use case for an event-driven system.<br />
<br />
One of the challenges with event-driven development is making sure that the event listeners are instantiated before the emitters begin sending events. An option for getting around this is to instantiate all listeners as part of the application's bootstraping/initialization process. But this can become cumbersome and have performance implications.<br />
<br />
Instead of always instantiating every listener, we can use our DIC and factory methods. Since event emitters are instantiated through a factory method, we use that same factory method to initialize listeners to any events that emitter may emit. By tying the creation of the emit to the creation of the listeners, we ensure that we only instantiate those listeners for which we have created an emitter.<br />
<br />
For example, we know that the HelloController may emit page view events, so we make sure that whenever we instantiate a HelloController, we also create a page view listener and set it to listen for the event.<br />
<br />
<script src="https://gist.github.com/4122568.js?file=3-events.php"></script><br />
<br />
<h3>Potential Issues</h3><br />
Several good questions were asked during my presentation, and I attempted to answer them as best as I could.<br />
<br />
<strong>Is debugging harder having the definitions of the objects away from their usage?</strong><br />
Since you are explicitly passing an object's dependencies, and you get all the dependencies from the same place, you can minimize external state that may affect an object's behavior. Also, by using factory methods, you can find exactly where an object was created. Without the DIC, an object or its dependencies could come from anywhere in your code. DI also makes unit- and regression-testing easier.<br />
<br />
<strong>Do the bootstrap/DIC definitions become unwieldy when you have a lot of interconnected objects?</strong><br />
The DIC is kept simple by having each factory method responsible for creating only one type of object. If each method is small and isolated, the file may become very large, but the logic in the file will remain easy to read. You can even break the bootstrap of the DIC into multiple files, each file handling a different component or section of the project's dependencies. Circular dependencies may be a problem with a highly connected object graph, but that is a separate problem regardless of using a DIC or not.<br />
<br />
<strong>Where is a good place to start with refactoring an existing large codebase to use DIC?</strong><br />
It's a multi-step process:<br />
<ol><li>Pick a class and create a factory method on a globally accessible DIC that instantiates that class.</li>
<li>For every place in the class that uses the "new" operator, replace it with a call to a factory function/method on the DIC.</li>
<li>Move calls to the factory methods into the class's constructor, and store the results as properties on the object.</li>
<li>Change the constructor to not call the factory methods itself, but receive its dependencies as parameters.</li>
<li>Update the class's factory method to call the other factory methods and pass in the dependencies to the constructor.</li>
<li>Find any place in the code that instantiates the class and replace it with a call to the class's factory method.</li>
</ol><br />
Repeat this process for any usage of the "new" operator in the code. By the end, the only place in the code that uses the "new" operator or refers to the DIC should be the DIC itself.<br />
<br />
<br />
<br />
Josh Adellhttp://www.blogger.com/profile/13146384900735324084noreply@blogger.com3tag:blogger.com,1999:blog-4037079221500014725.post-16262794594440434852012-09-27T12:00:00.000-04:002012-09-29T00:59:16.910-04:00Interfaces and Traits: A Powerful ComboIf you're not using <a href="http://php.net/manual/en/language.oop5.interfaces.php">interfaces</a> in PHP, you are missing out on a powerful object-oriented programming feature. An interface defines how to interact with a class. By defining an interface and then implementing it, you can guarantee a "contract" for consumers of a class. Interfaces can be used across unrelated classes. And they become even more useful when combined with the new <a href="http://php.net/manual/en/language.oop5.traits.php">traits</a> feature in PHP 5.4.<br />
<br />
Let's suppose we have a class called <code>User</code>. Users have an <code>Address</code> that our application mails packages to via a <code>PackageShipper</code>:<br />
<br />
<pre class="prettyprint lang-php">class Address {
// ... setters and getters for address fields ...
}
class User {
protected $address;
public function setAddress(Address $address) {
$this->address = $address;
}
public function getAddress() {
return $this->address;
}
// ... other user logic ...
}
class PackageShipper {
public function shipTo(User $user) {
$address = $user->getAddress();
// ... do shipping code using $address ...
}
}
</pre><br />
Our application is happily shipping packages, until one day a new requirement comes in that we need to be able to ship packages to Companies as well. We create a new <code>Company</code> class to handle this:<br />
<br />
<pre class="prettyprint lang-php">class Company {
protected $address;
public function setAddress(Address $address) {
$this->address = $address;
}
public function getAddress() {
return $this->address;
}
// ... other company logic ...
}
</pre><br />
Now we have a problem. The PackageShipper class only knows how to handle Users. What we need is for the PackageShipper class to handle anything that has an Address.<br />
<br />
We could make a base class that both Users and Companies inherit from and have the PackageShipper accept any class that descends from the base class. This is an unsatisfying solution. Semantically, Users and Companies are different entities, and there may not be enough common functionalty to move to a base class that both inherit. They may not have much in common besides the fact that they both have an address. Also, either class may already extend another class, and we cannot extend more than one class since PHP has a single-inheritance object model.<br />
<br />
Instead, we can use an interface to define the common parts of Users and Companies that PackageShipper needs to deal with. Then, Users and Companies can implement that interface, and PackageShipper only has to deal with objects that implement the interface.<br />
<br />
<pre class="prettyprint lang-php">inteface Addressable {
public function setAddress(Address $address);
public function getAddress();
}
class User implements Addressable {
protected $address;
public function setAddress(Address $address) {
$this->address = $address;
}
public function getAddress() {
return $this->address;
}
// ... other user logic ...
}
class Company implements Addressable {
protected $address;
public function setAddress(Address $address) {
$this->address = $address;
}
public function getAddress() {
return $this->address;
}
// ... other company logic ...
}
class PackageShipper {
public function shipTo(Addressable $entity) {
$address = $entity->getAddress();
// ... do shipping code using $address ...
}
}
</pre><br />
A class can implement many different interfaces, and classes with different bases can still implement the same interface.<br />
<br />
But there is still a problem. Both Company and User implement the <code>Addressable</code> interface using the same code. It is not required to do this; as long as the constraints of the interface are met, the implementation does not have to be the same for every class that implements the interface. But in this case, they both implement the interface in the same way, and we have duplicated code. If a third Addressable class came along, it is possible that it would also implement the interface the same way, leading to even more duplication.<br />
<br />
If you are using PHP 5.3 or less, there is little that can be done about this situation. But if you are using PHP 5.4, there is a new construct that is made to handle exactly these types of code duplication scenarios: <a href="http://php.net/manual/en/language.oop5.traits.php">traits</a>.<br />
<br />
A trait is similar to a class in that they both implement methods and have properties. The difference is that classes can be instantiated, but traits cannot. Instead, traits are added to class definitions, giving that class all the methods and properties that are defined in the trait.<br />
<br />
A helpful way to think about traits is like a macro: using a trait in a class is the same as if the code in the trait were copied into the class.<br />
<br />
Using traits, we can clean up the code duplication and still meet the contract defined by the Addressable interface:<br />
<br />
<pre class="prettyprint lang-php">trait AddressAccessor {
protected $address;
public function setAddress(Address $address) {
$this->address = $address;
}
public function getAddress() {
return $this->address;
}
}
class User implements Addressable {
use AddressAccessor;
// ... other user logic ...
}
class Company implements Addressable {
use AddressAccessor;
// ... other company logic ...
}
</pre><br />
Now, any class can immediately implement the Addressable interface by simply using the <code>AddressAccessor</code> trait. All the duplicated code has been moved to a single place.<br />
<br />
The trait itself does not implement Addressable. This is because only classes can implement interfaces. Remember that traits are nothing more than code that gets copied into a class, they are not classes on their own.<br />
<br />
Interfaces in PHP are a useful feature for enforcing code correctness. And when interfaces are combined with traits, they form a powerful tool which allows for faster development, decreased code duplication, greater readbility, and better maintainability.<br />
<br />
Here is all the final code for the package shipping application:<br />
<br />
<pre class="prettyprint lang-php">class Address {
// ... setters and getters for address fields ...
}
inteface Addressable {
public function setAddress(Address $address);
public function getAddress();
}
trait AddressAccessor {
protected $address;
public function setAddress(Address $address) {
$this->address = $address;
}
public function getAddress() {
return $this->address;
}
}
class User implements Addressable {
use AddressAccessor;
// ... other user logic ...
}
class Company implements Addressable {
use AddressAccessor;
// ... other company logic ...
}
class PackageShipper {
public function shipTo(Addressable $entity) {
$address = $entity->getAddress();
// ... do shipping code using $address ...
}
}
</pre>Josh Adellhttp://www.blogger.com/profile/13146384900735324084noreply@blogger.com2tag:blogger.com,1999:blog-4037079221500014725.post-77652483138428984522012-08-31T15:24:00.001-04:002012-09-28T19:00:38.684-04:00Plumbing PHPWhen building a project with <a href="http://neo4j.org/">Neo4j</a> or most other graph databases, it is impossible to avoid learning about <a href="http://tinkerpop.com/">Tinkerpop's</a> excellent <a href="http://gremlin.tinkerpop.com">Gremlin</a> graph processing language. The processing layer of Gremlin is built on top of <a href="http://pipes.tinkerpop.com/">Pipes</a>, a dataflow programming library.<br />
<br />
I was inspired by the syntax and ease-of-use of Gremlin to build a simple processing pipeline library in PHP. The result is <a href="https://github.com/jadell/plumber">Plumber</a>, a library for easily building extensible deferred-processing pipelines.<br />
<br />
Plumber is built on top of PHP's native <code><a href="http://php.net/manual/en/class.iterator.php">Iterator</a></code> library. The idea is simple: instantiate a new processing pipeline, attach processing pipes to it, then send an iterator through the pipeline and iterate over the results in a <code>foreach</code>. Each element that comes out the other end of the pipeline has been passed through an processed by each pipe. And because Plumber uses iterators, it natively supports lazy-loading and Just-In-Time evaluation.<br />
<br />
A simple example would be reading a set of records from a database, formatting them in some manner, then <code>echo</code>'ing them out to the screen:<br />
<pre class="prettyprint lang-php">$users = // code to retrieve user records from a database as an array or Iterator...
$names = array();
foreach ($users as $user) {
if (!$user['first_name'] || !$user['last_name']) {
continue;
}
$name = $user['first_name'] . ' ' . $user['last_name'];
$name = ucwords($name);
$name = htmlentities($name);
$names[] = $name;
}
// later on, display the names
foreach ($names as $name) {
echo "$name<br>";
}
</pre>There are a few obvious downsides to doing things this way: the entire set of records is looped through more than once; all the records must be in memory at the same time (twice even, once for $users and once for $names); and the processing steps in the <code>foreach</code> are executed immediately on every record. These may not seem like a big deal if the record set is small and the processing steps are trivial, but they can become big problems if you are not careful.<br />
<br />
Here is the same code using Plumber:<br />
<pre class="prettyprint lang-php">$users = // code to retrieve user records from a database as an array or Iterator...
$names = new Everyman\Plumber\Pipeline();
$names->filter(function ($user) {
return $user['first_name'] && $user['last_name'];
})
->transform(function ($user) {
return $user['first_name'] . ' ' . $user['last_name'];
})
->transform('ucwords')
->transform('htmlentities');
// later on, display the names
foreach ($names($users) as $name) {
echo "$name<br>";
}
</pre>The list of $users is only looped through one time, and there is no need to keep a separate list of $names in sync with the $users list. Each $user is transformed into a $name on-demand, keeping resources free.<br />
<br />
This can all be accomplished using Iterators, but there is quite a bit of boilerplate code involved. Plumber is meant to remove most of the boilerplate and let the developer concentrate on writing their business logic.<br />
<br />
There is more to Plumber, including several built-in pipe types, and the ability to extend the library with your own custom pipes. It is also not necessary to use the fluent interface, if that is not your style. More usage information can be found in the README file in the <a href="https://github.com/jadell/plumber">Plumber github repo</a>. Constructive feedback is always welcome!<br />
<br />
Josh Adellhttp://www.blogger.com/profile/13146384900735324084noreply@blogger.com6tag:blogger.com,1999:blog-4037079221500014725.post-22508317980994181782012-07-15T16:39:00.000-04:002012-07-15T16:39:16.694-04:00No Best Practices YetI'm getting a little tired of the constant onslaught of religious wars in software development these days. Every day, there's a new "PHP sucks vs. PHP is the best evar!", "Nodejs is crap vs Nodejs async FTW!", "OOP is a failure vs. FP doesn't get things done", "Agile vs. anything else" article. It's frustrating, because everyone is wrong.<br />
<br />
Mankind has been building bridges for literally tens of thousands of years. From the first moment a primitive monkey-man threw a log across a stream and every other moneky-man tribe copied the idea, we've been building bridges. And guess what? As a species, we're really good at it.<br />
<br />
And over the millennia, we've developed the discipline of <em>engineering</em>. At its core, engineering is nothing more than a set of <em>best practices</em> that say, "Hey, if you build your bridge this way or that way (depending on its purpose), it will most likely stay up for a really long time and continue to serve the function that you meant it to serve." If you ignore those best practices of engineering, your bridge may stay up, or it may not. And anyone who does build their bridges according to best practices will be able to look at your bridge and tell you why you have failed.<br />
<br />
How does this relate to software? Mankind has been writing computer code for less than a century. Modern programming languages have been around for about 40 years. The current crop of "high level" languages (PHP, Ruby, Java, Python, Javascript, Clojure, etc.) have only been around for about 20 years. The processes by which software gets built have been constantly evolving; we can't even agree on a definition of an "agile methodology." There are new programming languages, new frameworks, new architectures, new hardware, new ideas constantly coming out of the best minds in our field.<br />
<br />
So why does any single camp believe they know best when it comes to software "best practices?" Our discipline is so young that I don't believe any tool or method used by any software developer nowadays can be considered a "best practice."<br />
<br />
I'm sure back in the prehistoric days, monkey-tribes thought that a log across the stream was a best practice. They couldn't envision how mathematics, physics and materials would change to allow them to build better, stronger bridges. Fast-forward to current day, and building bridges is a pretty rigorous discipline. Even breakthroughs in stronger, lighter materials don't change the underlying principles of bridge-building. Engineers might have personal preferences for the type of bridges they like to build, but they don't debate over the fundamental rules of how to build a given bridge type, or which bridge type might be superior for a given purpose (unless the purpose is so trivial that aesthetics or personal preference can be a factor beyond physical properties.)<br />
<br />
And yet, every time a new language or framework pops up in software, entire populations of the development community are willing to abandon what came before in droves and proclaim the new way as self-evidently superior. Ignoring, of course, that that everyone said the same thing about whatever they were leaving behind the last time.<br />
<br />
Every article I've read on either side of any of these debates can be boiled down to "my brain doesn't work that way, it works this way, therefore, this way is superior." I've yet to see a compelling argument given for any side of any software debate that doesn't have the subtext of simply being the author's personal preference. And that right there tells me that our profession is too young to have decided on what is or is not a best practice.<br />
<br />
So yeah, let's keep throwing our logs across the stream and fighting over whether oak is better than pine, ignoring that as soon as we discover steel the debate will be pointless. At least be willing to admit that while the tools and methods we have today might be the best we can come up with, they're certainly not "best practices" yet.Josh Adellhttp://www.blogger.com/profile/13146384900735324084noreply@blogger.com2tag:blogger.com,1999:blog-4037079221500014725.post-42477151225557041282012-05-14T00:29:00.002-04:002014-02-15T13:19:28.419-05:00Neo4jPHP Available as a Composer Package<a href="http://github.com/jadell/neo4jphp">Neo4jPHP</a> is now available as a <a href="http://getcomposer.org/">Composer</a> package. To use it in your project, create a "composer.json" file in the root of your project, with the following contents:<br />
<pre class="prettyprint lang-js">{
"require":{
"everyman/neo4jphp": "dev-master"
}
}
</pre>After installing Composer, run the following command in the root of the project:<br />
<pre class="prettyprint lang-sh">> php composer.phar install
</pre>The Neo4jPHP library will be downloaded and installed in your project under "vendor/everyman/neo4jphp". An autoloader will be created at "vendor/autoload.php" which you should include in your project, then start using Neo4jPHP as normal:<br />
<pre class="prettyprint lang-php">require("vendor/autoload.php");
$client = new Everyman\Neo4j\Client();
$node = $client->makeNode();
</pre>To get the latest version of Neo4jPHP, run:<br />
<pre class="prettyprint lang-sh">> php composer.phar update
</pre>Josh Adellhttp://www.blogger.com/profile/13146384900735324084noreply@blogger.com0tag:blogger.com,1999:blog-4037079221500014725.post-19552745477036950052012-04-23T21:22:00.001-04:002012-04-23T21:22:28.598-04:00Runtime Expectations Javascript ThrowdownLast week, <a href="http://www.meetup.com/Triangle-JavaScript/">TriangleJS</a> were the guests on the talk-show podcast <a href="http://blog.runtimeexpectations.com/rte/2012/04/23/runtime-expectations-episode-21-javascript-framework-discussion-with-trianglejs/">Runtime Expectations</a>. The topic was a Javascript framework "throwdown" which covered a wide range of different frameworks and toolkits. We even discussed if frameworks were necessary at all. <a href="https://twitter.com/#!/bfarrellforever">Ben</a> and <a href="https://twitter.com/#!/adrianpomilio">Adrian</a> are great guys and it was a lot of fun being on their show and hanging out for beers afterwards.<br />
<br />
If you want to hear me make a total stuttering ass out of myself, I start rambling around 23:34 then fade out so more informed people can talk.Josh Adellhttp://www.blogger.com/profile/13146384900735324084noreply@blogger.com1tag:blogger.com,1999:blog-4037079221500014725.post-24663365834720932332012-04-02T08:00:00.000-04:002012-04-02T09:26:14.397-04:00Data Mapper Injection in PHP ObjectsI was trying to figure out a simple way to have data from a database injected into a domain object in PHP. I don't want to enforce that the domain object implements setters and getters for every property, and the burden on the developer for implementing the data injection interface should be as low as possible.<br />
<br />
My solution relies on PHP's ability to pass and assign values by reference. The data mapper is injected into domain object. The domain object tells the mapper to attach to the object's properties. The mapper can then read and write to those properties as if they were its own.<br />
<pre class="prettyprint lang-php">interface Mappable {
public function injectMapper(Mapper $mapper);
}
class Mapper {
protected $attached = array();
public function attach(&$ref, $fieldName) {
$this->attached[$fieldName] = &$ref;
}
public function fromDataStore($data) {
foreach ($data as $fieldName => $value) {
if (array_key_exists($fieldName, $this->attached)) {
$this->attached[$fieldName] = $value;
}
}
}
public function fromDomainObject() {
$data = array();
foreach ($this->attached as $fieldName => $value) {
$data[$fieldName] = $value;
}
return $data;
}
}
class Product implements Mappable {
protected $id;
protected $name;
public function injectMapper(Mapper $mapper) {
$mapper->attach($this->id, 'id');
$mapper->attach($this->name, 'name');
}
// ...domain/business logic methods go here...
}
// Create a product and manipulate it
$product = new Product();
$product->doFoo();
$product->linkToBar();
// Inject the data mapper
$mapper = new Mapper();
$product->injectMapper($mapper);
// This could be a call to save the array of data in a database
print_r($mapper->fromDomainObject());
// Read a product from the "database"
$product = new Product();
$mapper = new Mapper();
$product->injectMapper($mapper);
$mapper->fromDataStore( array("id"=>123, "name"=>"Widgets") );
</pre>So what's happening here? The fields of the Product domain object are passed into the Mapper's `attach` method by reference. This allows that method to modify the variable passed in without passing it back to the caller.<br />
<pre class="prettyprint lang-php">public function attach(&$ref, $fieldName) {
$this->attached[$fieldName] = &$ref;
}
</pre>If the value were only passed by reference, only the `attach` method would be able to modify it. As soon as the method returned, the reference would be lost. In order to continue having a reference to the Product's properties, the `attach` method holds the reference in an array using "assign by reference."<br />
<br />
Note the ampersand after the equals sign. This tells PHP that we don't want to assign a copy of the value in $ref, we want the reference to $ref itself. This allows us to later modify the value of the property in Product by modifying the value of a reference to the same place in memory that that Product's properties are held.<br />
<br />
So now we have the beginnings of a basic data mapper that remains loosely coupled to our domain objects, and without having to write an explicit mapping class for each domain object-to-database mapping.Josh Adellhttp://www.blogger.com/profile/13146384900735324084noreply@blogger.com0tag:blogger.com,1999:blog-4037079221500014725.post-4889997279757539762012-03-04T22:06:00.001-05:002012-03-05T11:38:46.723-05:00GetSet Methods vs. Public PropertiesI was recently having a debate with a coworker over the utility of writing getter and setter methods for protected properties of classes. On the one hand, having getters and setters seems like additional boilerplate and programming overhead for very little gain. On the other hand, exposing the value properties of a class seems like bad encapsulation and will overall lead to code that is more difficult to maintain.<br />
<br />
I come down firmly on the get/set method side of the fence. It takes very little extra code, defines the interface to the class explicitly and makes the class easier to extend and maintain in the long run. Let's try an example (in PHP, though the premise holds for any language with public/private properties.)<br />
<pre class="prettyprint lang-php">class PublicValues {
public $foo;
public $bar;
}
class GetSet {
protected $foo;
protected $bar;
public function getFoo() {
return $this->foo;
}
public function setFoo($value) {
$this->foo = $value;
}
public function getBar() {
return $this->bar;
}
public function setBar($value) {
$this->bar = $value;
}
}
$public = new PublicValues();
$public->foo = array('some'=>'value', 'data'=>'structure');
$public->bar = 123;
$getset = new GetSet();
$getset->setFoo(array('some'=>'value', 'data'=>'structure'));
$getset->setBar(123);
echo "Public is: ".$public->bar." and ".print_r($public->foo, true)."\n";
echo "GetSet is: ".$getset->getBar()." and ".print_r($public->getFoo(), true)."\n";
</pre>Obviously, class `PublicValues` is much shorter than class `GetSet`. Shorter is better, right? Less code to maintain, fewer places for bugs to creep in. And it didn't take me nearly as long to type (though, truthfully, `GetSet` didn't take that long to write, either, as it was mostly cut-and-paste.)<br />
<br />
Now let's say that now we have decided to wrap the array data structure in a new `Widget` class and we need to ensure that `foo` can only be of type `Widget`. Modify the classes to make this so:<br />
<pre class="prettyprint lang-php">class PublicValues {
// ... add a whole new method
public function setFoo(Widget $value) {
$this->foo = $value;
}
}
class GetSet {
// ... only change here is type-hinting
public function setFoo(Widget $value) {
$this->foo = $value;
}
// ...
}
// ...
$public->setFoo(new Widget(array('some'=>'value', 'data'=>'structure')));
// ...
$getset->setFoo(new Widget(array('some'=>'value', 'data'=>'structure')));
// ...
</pre>`PublicVars` had an entirely new method added to it. `GetSet` only had a small modification to an existing method. <br />
<br />
Actually, this still isn't perfect; application code can still say `$public->foo = 'whateverIWant';` without going through the `PublicVars::setFoo` method. <em>In fact, while writing this example, I forgot until my third revision to go back and change the sample code to use the set method.</em> Let's make sure that doesn't happen in the future:<br />
<pre class="prettyprint lang-php">class PublicValues {
// ... change foo so that client code can't set it to the wrong thing
protected $foo;
// ... still need to be able to read foo, so now we add a getter
public function getFoo() {
return $this->foo;
}
// ...
}
// GetSet remains the same ...
// ...
$public->setFoo(new Widget(array('some'=>'value', 'data'=>'structure')));
// ...
echo "Public is: ".$public->bar." and ".print_r($public->getFoo(), true)."\n";
</pre>Notice that `GetSet` didn't have to change at all in this case, and the only client code that had to change for it was searching for anywhere calling `GetSet::setFoo` and making sure it provided a `Widget`.<br />
<br />
As for `PublicValues`, we'll have to go through all the code making sure that everywhere that was setting `foo` with assignment now calls the set method (with a proper `Widget`), and anywhere that was reading it directly calls the get method.<br />
<br />
My future self is screaming at my current self. Now we have two ways of accessing the PublicValues class: one way through direct value reading and writing (bar), and the other through getset methods (foo). As a consumer of this class, I have no way of knowing which properties are openly accessible and which are guarded through getset methods, other than going back and looking through the code.<br />
<br />
To solve this problem, I am going to use a method that I see in almost every PHP project of non-trivial size that I have ever worked with or seen the source code of: use magic __get and __set to determine if the property is directly accessible or not:<br />
<pre class="prettyprint lang-php">class PublicValues {
// ...
public __get($name) {
$method = 'get'.$name;
if (method_exists($this,$method)) {
return $this->$method();
} else {
return $this->$name;
}
}
public __set($name, $value) {
$method = 'set'.$name;
if (method_exists($this,$method)) {
return $this->$method($value);
} else {
$this->$name = $value;
}
}
}
</pre>As before, `GetSet` doesn't change one bit. For `PublicValues`, the rest of the code can continue to use what looks like straight assignment, and if I want to know if the assignment is direct modification or a magic set, I only need to look in only two places to figure it out: look for a named getset method, or assume I am using the magic __get and __set. Perfect...<br />
<br />
...until we have to add something to this class that absolutely must <strong>not</strong> be modified by client code.<br />
<pre class="prettyprint lang-php">class PublicValues {
// ...
protected $modificationToThisCrashesTheApp;
// ...
}
class GetSet {
// ...
protected $modificationToThisCrashesTheApp;
// ...
}
</pre>Now we have a big problem. Clients of `GetSet` are protected, because nothing in `GetSet` can be modified except through a class method. We don't provide a method for accessing the dangerous property, so clients can't hurt themselves.<br />
<br />
But clients of `PublicValues` can modify any property they want. The magic __set method makes public any protected value that does not have an explicit set method. We could fix this by adding a set method that throws an exception. Or we could solve it by blacklisting that property in the magic __set. Or we could make sure everyone (including new hires six months from now) knows not to touch this property. Or we could solve it by...<br />
<br />
Truthfully, it doesn't matter how you solve it. Encapsulation of `PublicValues` has been broken from the beginning. We haven't even tried adding validation to either class, or making the setting of one value affect another, or adding read- or write-only methods. And over the course of just a few changes, we've had to add get/set methods anyway. We've just done it later in the application lifecycle after client code is already using the class, the time when it is most costly to make those sorts of changes.<br />
<br />
In a blog post, it's easy to see these changes coming in a few paragraphs and account for them mentally. But imagine this in a large system, one that is constantly having features added, bugs fixed and maintenance performed over the course of months or years. Every change in this example could have come months apart. I don't expect myself or my fellow programmers to keep track of the intricacies of a single class over that timeframe, especially if that class is modified rarely, is one class in hundreds and we've been working in thousands of lines of other code in the meantime.<br />
<br />
So do yourself a favor and take the 20 seconds to wrap your properties in boilerplate getset methods from the start. Not because it's a best practice, but because future you will be grateful.<br />
<br />
Here is the meme-pic that started the debate:<br />
<br />
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEii13Ot5FMen1t0lvkl0w81doRGLGsUnT7HTSc32QoHoLhwC_NH8zZxsN8oEY8v9392Q0g9Mlq7fswE31Q-9V2FguP1iskfPGqrCh2IsVKGXfPybfupmLnZ7susUbOljjMID3Jim1Bd0nq2/s1600/The-Most-Interesting-Man-in-the-World.jpg" imageanchor="1" style=""><img border="0" height="320" width="255" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEii13Ot5FMen1t0lvkl0w81doRGLGsUnT7HTSc32QoHoLhwC_NH8zZxsN8oEY8v9392Q0g9Mlq7fswE31Q-9V2FguP1iskfPGqrCh2IsVKGXfPybfupmLnZ7susUbOljjMID3Jim1Bd0nq2/s320/The-Most-Interesting-Man-in-the-World.jpg" /></a></div><br />
<br />Josh Adellhttp://www.blogger.com/profile/13146384900735324084noreply@blogger.com6tag:blogger.com,1999:blog-4037079221500014725.post-6698655518669774382012-02-23T23:42:00.001-05:002012-02-25T12:01:26.826-05:00Similarity-based Recommendation EnginesI am currently participating in the <a href="http://neo4j-challenge.herokuapp.com/">Neo4j-Heroku Challenge</a>. My entry is a -- as yet, unfinished -- beer rating and recommendation service called <a href="https://frostymug.herokuapp.com/">FrostyMug</a>. All the major functionality is complete, except for the actual recommendations, which I am currently working on. I wanted to share some of my thoughts and methods for building the recommendation engine.<br />
<br />
My starting point for the recommendation engine was a blog post by Marko Rodriguez, <a href="http://markorodriguez.com/2011/09/22/a-graph-based-movie-recommender-engine/">A Graph-Based Movie Recommender Engine</a>. Marko's approach is pretty straightforward and he does an excellent job of explaining the subtleties of collaborative and content-based filtering. It was also a great way to get acquainted with the <a href="http://gremlin.tinkerpop.com/">Gremlin graph traversal API</a>, which I have been unwisely avoiding for fear of it being too complicated.<br />
<br />
Basic collaborative filtering works like this: if I rate a beer, say, <a href="http://www.loneriderbeer.com/beers/">Lonerider Sweet Josie Brown Ale</a> as 9 out of 10, and you rate that same beer as 8 out of 10, then I should be able to get recommendations based on other beers that you (and others who have rated that beer similarly to me) have rated highly.<br />
<br />
I think we can improve on this. You and I happen have rated one beer the same way. What happens if that is the only beer we have rated similarly? What if I have rated <a href="http://www.fullsteam.ag/beer/workers-comp/">Fullsteam Carver Sweet Potato Ale</a> at 8 out of 10, but you have rated it 2 out of 10? You have shown that you have poor taste, so why should I get recommendations based on your other ratings, simply because we happened to rate a single beer similarly?<br />
<br />
With a small modification, we can overcome this problem. Instead of basing recommendations off of one similar rating, I can calculate how similarly you and I rated all the things we have rated, and only get recommendations from you if I have determined we are similar enough in our tastes. Call this "similarity-based collaborative filtering."<br />
<br />
Finding similar users works like this:<br />
<ol><li>Find all users who have rated the same beers as I have.</li>
<li>For each rated beer, calculate the differences between my rating and that user's rating.</li>
<li>For each user, average the differences.</li>
<li>Return all users where the average difference is less than or equal to some tolerance (2 in the below example.)</li>
</ol><br />
Let's see what this looks like in Gremlin, a domain specific graph traversal language in Groovy. Gremlin is built on the concept of "pipes" where a single chain of operations runs from beginning to end on each element put into the pipe. The elements in our case are the user nodes, beer nodes and rating edges that connect them in our graph:<br />
<pre class="prettyprint lang-java">// We're going to use this again and again, so make a single "step" for it,
// to which we can pass a "similarity threshold"
Gremlin.defineStep('similarUser', [Vertex,Pipe], {Integer threshold ->
// Capture the incoming pipe
target = _();
// Set up a table to hold ratings for averages
m=[:].withDefault{[0,0]};
// Find all beers rated by the target, and store each rating
target.outE("RATED").sideEffect{w=it.rating}
// For each rated beer, find all the users who have also rated that beer
// and who are not the target user, then go back to their ratings
.inV.inE("RATED").outV.except(target).back(2)
// At this point in the pipeline, we are iterating over
// only the ratings of beers that both users have rated.
// For each rating, calculate the difference between
// the target user's rating and the compared user's rating
.sideEffect{diff=Math.abs(it.rating-w)}
// For each compared user, store the number of beers rated in common
// and the total difference for all common ratings.
.outV.sideEffect{ me=m[it.id]; me[0]++; me[1]+=diff; }.iterate();
// Find the average difference for each user and filter
// out any where the average is greater than two.
m.findAll{it.value[1]/it.value[0] <= threshold}.collect{g.v(it.key)}._()
});
// Return all the users who are similar to our target
// with an average similarity of 2 or less
g.v(123).similarUser(2)
</pre>The result of this script is a list of all user nodes that are considered "similar" to the target user. Now we can get more accurate recommendations, based on users whose tastes are similar to ours. Given users who are similar to me, recommend beers that they rated highly: <br />
<ol><li>Find all beers rated by similar users.</li>
<li>For each beer, average the similar users' ratings.</li>
<li>Sort the beers by highest to lowest average and return the top 25.</li>
</ol><pre class="prettyprint lang-java">r=[:].withDefault{[0,0]};
// Find similar users
g.v(123).similarUser(2)
// Find all outgoing beer ratings by similar users
// and the beer they are rating
.outE("RATED").sideEffect{rating=it.rating}.inV
// Store the ratings in a map for averaging
.sideEffect{me=r[it.id]; me[0]++; me[1]+=rating; }
// Transform the map into a new map of beer id : average
r.collectEntries{key, value -> [key , value[1]/value[0]]}
// sort by highest average first, then take the top 25
.sort{a,b -> b.value <=> a.value}[0..24]
// Transform the map back into a mapping of beer node and rating
.collect{key,value -> [g.v(key), value]}
</pre>Now I only get recommendations from users whose tastes are similar to mine.<br />
<br />
I can also calculate an estimated rating for any beer that I have not rated. To see how that might be useful, consider Netflix. When you look at any movie on Netflix that you have not already rated, they show you their best guess for how you would like that movie. This can be helpful in determining which of a set of movies you want to watch, or in our case, which beer you might want to try given limited pocket money. Again, we can use our similar users to figure out the estimated rating of a given beer: <br />
<ol><li>Find all ratings of the target beer by similar users.</li>
<li>Return the average of their ratings of that beer.</li>
</ol><pre class="prettyprint lang-java">// Grab our target beer
beer=g.v(987)
// Find similar users
g.v(123).similarUser(2)
// For each similar user, find all that have rated the target beer
.outE("RATED").inV.filter{it==beer}
// Go back to their ratings, and average them
.back(2).rating.mean()
</pre>So now we have a simple recommendation engine based on ratings by users with similar tastes. There are a few improvements that could still be made: <br />
<ul><li>Don't consider a user "similar" unless they have rated at least 10 beers in common (or some other threshold). This prevents users who have only rated a few beers in common from being falsely labeled as similar.</li>
<li>Calculate the similar users ahead of time, and store them with a "SIMILAR" relationship to the target user. Periodically recalculate the similar users to keep the list fresh.</li>
<li>Use random sampling or size limiting on the number of users and ratings checked when determining similarity, trading accuracy for performance.</li>
</ul>Many thanks to Marko for his help on the mailing list, and with giving me some pointers in writing this, and for being a pretty awesome guy in general.<br />
<br />
Update: Luanne Misquitta posted about how to do <a href="http://thought-bytes.blogspot.in/2012/02/similarity-based-recommendations-with.html">similarity-based recommendations with Cypher</a>. Thanks, Luanne!<br />Josh Adellhttp://www.blogger.com/profile/13146384900735324084noreply@blogger.com3tag:blogger.com,1999:blog-4037079221500014725.post-45121910091240799372012-01-14T00:21:00.004-05:002012-01-17T01:14:35.505-05:00Command Invoker Pattern with the Open/Closed PrincipleOver on DZone, <a href="http://twitter.com/#!/giorgiosironi">Giorgio Sironi</a> demonstrates the <a href="http://css.dzone.com/articles/openclosed-principle-real">"Open/Closed Principle on real world code"</a>. The pattern demonstrated is similar to the <a href="http://en.wikipedia.org/wiki/Command_pattern">Command Pattern</a>, and the post does a good job of introducing a class that is <em>open</em> for extension but <em>closed</em> for modification. Giorgio mentions that there is still more work to be done. I thought I might take a stab at creating a flexible and extendable command invocation solution.<br />
<br />
One of the issues that should be addressed is that the invoker (the Session class in the post), is only extensible by actually extending the class. If we want to add custom methods to it, we have two options: make our own sub-class, which should be avoided when favoring composition over inheritance; or modifying the base class's constructor, which violates the open/closed principle.<br />
<br />
Another thing that could be improved is that the invoker is violating the <a href="http://en.wikipedia.org/wiki/Single_responsibility_principle">Single-Responsibility principle</a> (which violates the "S" in SOLID.) It needs to know not just how to execute commands, but also how to construct them. As the list of available commands grows, and the constructors for each command become more complex, the invoker needs to contain more logic to build each command type.<br />
<br />
Let's overcome some of these issues, and also make the code even more extensible. I'll use a simplified command invoker to demonstrate.<br />
<br />
To start, let's remove the complexity of requiring the invoker know how to build each command object. A simple way to do this is the move the factory methods into the individual command classes. We'll also define a simple interface to ensure that our commands are executable. Here are two simple commands. Note that the constructor is not part of the interface, which allows the command classes to have completely different constructor method signatures.<br />
<pre class="prettyprint lang-php">interface Command {
public static function register();
public function execute();
}
class HelloCommand implements Command {
protected $name;
public static function register() {
return function ($name) {
return new HelloCommand($name);
};
}
public function __construct($name) {
$this->name = $name;
}
public function execute() {
echo "Hello, " . $this->name . "\n";
}
}
class PwdCommand implements Command {
public static function register() {
return function () {
return new PwdCommand();
};
}
public function execute() {
echo "You are here: " . getcwd() . "\n";
}
}
</pre>Each command class has a static `register` method that returns an anonymous function which knows how to instantiate the class. We'll call this the "factory" function. In the example, the factory function signatures are the same as the constructors, but that is not necessary; as long as the factory function can construct a valid object, it can take whatever parameters it needs.<br />
<br />
Our commands can construct themselves, so the invoker only has to know how to map its own method calls to the commands. Here is the simplified Invoker class:<br />
<pre class="prettyprint lang-php">class Invoker {
protected $commandMap = array();
public function __construct() {
$this->register('hello', 'HelloCommand');
$this->register('pwd', 'PwdCommand');
}
public function __call($command, $args) {
if (!array_key_exists($command, $this->commandMap)) {
throw new BadMethodCallException("$command is not a registered command");
}
$command = call_user_func_array($this->commandMap[$command], $args);
$command->execute();
}
public function register($command, $commandClass) {
if (!array_key_exists('Command', class_implements($commandClass))) {
throw new LogicException("$commandClass does not implement the Command interface");
}
$this->commandMap[$command] = $commandClass::register();
}
}
</pre>The important bit is the `register` method. It checks that the registered class is indeed a Command, and thus, that it will have both an `execute` and a `register` method. We then invoke the `register` method, and store the factory function for that command.<br />
<br />
In the magic `__call` method, we have to use `call_user_func_array` since we don't know the real signature of the called factory function. If we wanted, we could have the `__call` method return the command object instead of executing it. That would turn the Invoker into an Abstract Factory for command objects.<br />
<br />
The main point is that Invoker never cares exactly how commands are constructed. Invoker now only has one responsibility: invoking commands.<br />
<br />
We execute commands by calling Invoker methods with parameters that match the factory function signature of the command being executed:<br />
<pre class="prettyprint lang-php">$invoker = new Invoker();
$invoker->hello('Josh');
$invoker->pwd();
</pre>By making the Invoker's `register` method public, we've also opened up the Invoker to allow a client to extend its functionality without having to modify the Invoker itself:<br />
<pre class="prettyprint lang-php">class CustomCommand implements Command {
protected $foo;
protected $bar;
public static function register() {
return function ($foo, $bar) {
return new CustomCommand($foo, $bar);
};
}
public function __construct($foo, $bar) {
$this->foo = $foo;
$this->bar = $bar;
}
public function execute() {
echo "What does it all mean? " . ($this->foo + $this->bar) . "\n";
}
}
$invoker->register('custom', 'CustomCommand');
$invoker->custom(40, 2);
</pre>We can even let clients overwrite existing commands by re-registering with a custom class:<br />
<pre class="prettyprint lang-php">$invoker->register('pwd', 'CustomCommand');
$invoker->pwd(2, 40);
</pre>If we don't want to allow this, we can build more checks into `register` that prevent overwriting built-in commands.<br />
<br />
So now we have an extendable, flexible command invocation system that requires no modification to the actual command runner. This could also be used as the basis for a plug-in system.<br />
<br />
Full sample code is available at <a href="http://gist.github.com/1610148">http://gist.github.com/1610148</a><br />
<br />
<br />
<br />
<br />
<br />Josh Adellhttp://www.blogger.com/profile/13146384900735324084noreply@blogger.com2tag:blogger.com,1999:blog-4037079221500014725.post-21578567649570106242012-01-03T16:11:00.000-05:002012-01-03T16:11:19.021-05:00Funkatron's MicroPHP ManifestoJust saw this over on <a href="https://twitter.com/#!/funkatron">@funkatron</a>'s blog:<br />
<br />
<a href="http://funkatron.com/posts/the-microphp-manifesto.html">The MicroPHP Manifesto</a>:<br />
<br />
<strong>I am a PHP developer</strong><br />
<ul><li>I am not a Zend Framework or Symfony or CakePHP developer</li>
<li>I think PHP is complicated enough</li>
</ul><br />
<strong>I like building small things</strong><br />
<ul><li>I like building small things with simple purposes</li>
<li>I like to make things that solve problems</li>
<li>I like building small things that work together to solve larger problems</li>
</ul><br />
<strong>I want less code, not more</strong><br />
<ul><li>I want to write less code, not more</li>
<li>I want to manage less code, not more</li>
<li>I want to support less code, not more</li>
<li>I need to justify every piece of code I add to a project</li>
</ul><br />
<strong>I like simple, readable code</strong><br />
<ul><li>I want to write code that is easily understood</li>
<li>I want code that is easily verifiable</li>
</ul><br />
I agree that there are "and-the-kitchen sink" frameworks out there that do too much. I also agree that having every developer re-invent the wheel is not the ideal situation. The main problem I see with the macro-frameworks is that they tend to not allow me to strip out their functionality and replace it with another library <em>when I need to</em>. I much prefer smaller libraries and components that I can mix and match if necessary. Isn't that the whole point of code re-use in the first place? <br />
<br />
So maybe this isn't so much about huge frameworks being bad as it is that they should be built off small, interchangeable coponents? Wasn't there <a href="http://pooteeweet.org/blog/2008">a call for more inter-operability</a> <a href="http://blog.astrumfutura.com/2011/10/interfacing-the-php-world-would-be-good/">a little while back</a>? <br />
<br />
Funkatron has made it pretty clear that these are his personal preferences, but that hasn't stopped <a href="http://www.reddit.com/r/PHP/comments/o19zj/the_microphp_manifesto/">the haters</a>.Josh Adellhttp://www.blogger.com/profile/13146384900735324084noreply@blogger.com0tag:blogger.com,1999:blog-4037079221500014725.post-52562610339684463512011-12-25T14:58:00.000-05:002011-12-25T15:57:10.679-05:00PHP Fog QuickstartFor a while, I've been trying to find a good hosting service to use for testing out PHP apps and ideas I have. My criteria was to find a service that would be like <a href="http://www.heroku.com/">Heroku</a> for PHP apps: simple to get started, pre-configured with access to a database, scalable on demand, and no-hassle deployment (able to integrate with continuous deployment.)<br />
<br />
Funnily enough, if you do a Google search for "heroku php" the first thing that comes up is <a href="http://phpfog.com/">PHP Fog</a>. They provide hosting for up to 3 projects for free on shared hosting. They also offer MongoDB and application monitoring as add-ons.<br />
<br />
Being it was Christmas and all, I decided to give myself a present and sign up. I was very surprised by how easy it was to get up and running. I managed to build a simple "echo" service in about 8 minutes, following roughly these steps:<br />
<br />
After logging in, click "Launch New App". This brings you to a screen where you can choose from several pre-built applications (popular CMS or blogging platforms) or skeleton projects built on the most popular PHP frameworks.<br />
<br />
For my purposes, I chose "Custom App". This will build an application containing exactly one "Hello world" PHP file. Enter a password for the MySQL database that will be set up, and a subdomain. The subdomain must be unique among all PHP Fog apps, and the setup won't continue until it is. After your app is set up, it can be given a custom domain name from $5 per month.<br />
<br />
Click "Create App". This will bring you to your project's management page. From here, you can find everything you need to start developing your app. The git repository address for the app is displayed, as are the database credentials, helpfully wrapped in a mysql_connect function call.<br />
<br />
Eventually, the "Status" light will turn green, indicating that your app is available. Clicking "View Live Site" brings you to your application, live to the world. Not much there. Time to change that.<br />
<br />
Follow the instructions to add an SSH key to PHP Fog, which will give you access to write to your application's git repo. Then use the given git address to clone the repo to your development environment. Also, grab the <a href="http://silex.sensiolabs.org/">Silex</a> microframework to start building the app:<br />
<pre class="prettyprint lang-sh">> git clone git@git01.phpfog.com:myapp.phpfogapp
> cd myapp.phpfogapp
> wget http://silex.sensiolabs.org/get/silex.phar
</pre>Edit index.php to provide a simple endpoint which will greet users:<br />
<pre class="prettyprint lang-php"><?php
require_once __DIR__.'/silex.phar';
$app = new Silex\Application();
$app->get('/', function() use($app) {
return 'Checkout <a href="/hello/world">Hello...</a>';
});
$app->get('/hello/{name}', function($name) use($app) {
return 'Hello '.$app->escape($name);
});
$app->run();
</pre>Also, create an .htaccess file that will redirect all traffic to index.php:<br />
<pre class="prettyprint">RewriteEngine On
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^ index.php [L]
</pre>Add the files to repository, commit them, then push the master branch to make the changes live:<br />
<pre class="prettyprint lang-php">> git add index.php .htaccess silex.phar
> git commit
> git push origin master
</pre>Reload your site to see the changes. Follow the "Hello" link, and play with the URL to see the results.<br />
<br />
After the holidays, I hope to actually build something interesting on the platform.<br />Josh Adellhttp://www.blogger.com/profile/13146384900735324084noreply@blogger.com6tag:blogger.com,1999:blog-4037079221500014725.post-66428236538169473872011-12-17T18:56:00.000-05:002011-12-17T18:57:39.689-05:00Decode JSON in Bash with PHPI recently found myself needing to make cUrl calls from the command-line to an endpoint which returned JSON responses. Rather than parsing through the JSON as a string, or downloading some third party tool to format it for me, I created this handy Bash alias that decodes JSON from the command-line using PHP.<br />
<br />
Put the following in your ~/.bashrc file:<br />
<pre class="prettyprint lang-sh">alias json-decode="php -r 'print_r(json_decode(file_get_contents(\"php://stdin\")));'"
</pre><br />
Usage from any process:<br />
<pre class="prettyprint lang-sh">echo '{"foo":"bar","baz":"qux"}' | json-decode
</pre><br />
Or from a cUrl call:<br />
<pre class="prettyprint lang-sh">curl -s http://example.com/json/endpoint | json-decode
</pre>Josh Adellhttp://www.blogger.com/profile/13146384900735324084noreply@blogger.com1tag:blogger.com,1999:blog-4037079221500014725.post-6329257226945323632011-12-08T17:59:00.001-05:002011-12-09T01:15:37.671-05:00Codeworks '11 Raleigh RundownI had a great time at <a href="http://codeworks.phparch.com/">CodeWorks 2011</a> in Raleigh this week, put on by the great folks at <a href="http://www.phparch.com/">php|architect</a>. Here is a rundown of the presentations, with some of my thoughts.<br />
<br />
<h2>CI:IRL</h2><a href="http://twitter.com/e3BethT">Beth Tucker Long</a> started out the day with a talk about continuous integration tools for automating away the pain of building and deploying PHP projects. One of her main themes was the importance of standardization in your code to avoid wasting mental effort on things like variable and class naming, which lines to put curly braces on, or how many spaces are in a single indent (editor: the answer is none; use a <b>tab</b> character.) The same can be said about manual testing and deployment processes.<br />
<br />
The tools discussed included <a href="http://pear.php.net/package/PHP_CodeSniffer">PHP_CodeSniffer</a>, the venerable and nigh ubiquitous <a href="http://www.phpunit.de">PHPUnit</a>, and multiple options for automated builds and testing (<a href="http://cruisecontrol.sourceforge.net/">CruiseControl</a> and <a href="http://jenkins-ci.org/">Jenkins/Hudson</a>.) Beth also talked about <a href="http://www.phing.info">Phing</a>, which is one of my new favorite tools. She also covered tools for automated documentation and report generation. I would have gone with <a href="http://www.docblox-project.org/">DocBlox</a> instead of <a href="http://www.phpdoc.org/">phpDocumentor</a>, but that's just a personal preference. The most useful thing about most of these tools is that many of them are aware of each other, directly or through plugins, and can be chained together to build a flexible CI pipeline.<br />
<br />
Most of the tools I had already heard of or am actively using. In addition to PHP_CodeSniffer for coding standards compliance, I would also throw in one or both of <a href="http://phpmd.org/">PHP Mess Detector</a> and <a href="https://github.com/sebastianbergmann/phpcpd">PHP Copy-Paste Detector</a>. These three tools together give great insights into the current state of your code base and its future maintainability, which Beth mentioned when discussing "technical debt".<br />
<br />
An important thing that Beth stressed at the end of her presentation was that not every project requires the use of every tool. It's important to take into account things like size/complexity of the project, timeline, budgets and team size. I think this was a valuable piece of advice that a lot of people forget. If you're anything like me, when you first learn about a new tool your response is to go out and find as many ways to use it as possible. I'm glad Beth ended with the reminder to keep in mind the constraints of your project when picking which tools to apply.<br />
<br />
<h2>How Beer Made Me A Better Developer</h2>Next up was <a href="http://twitter.com/jason_austin">Jason Austin</a>, who is one of the coordinators of my local <a href="http://www.meetup.com/mysql-144/">PHP meetup group</a>. Jason's talk was a call for all developers to find their passion. His two passions are beer and PHP, so he decided to combine them into a side project, and <a href="http://www.pintlabs.com/">Pint Labs</a> was born. All developers should find a side project related to their passion. It is something he looks for as a hiring manager. They also prevent burnout in your day job.<br />
<br />
Side projects give developers an opportunity to experiment without the consequences of failure that usually accompany employed or contract work. More than that, they allow for greater opportunities to bring new knowledge to a paid job. "The dumbest thing a manager can do is stop their developers from working on their own side projects," was one of my favorite lines from the presentation.<br />
<br />
I can't agree more, and I'm lucky to have never worked in an environment where an employer has either forbidden me from having side projects, or claimed that anything I work on in my own time belongs to the company. I learned at lunch that many people aren't so fortunate. I would be very wary of any employment where that was the case.<br />
<br />
As for side projects, even if you don't start your own, there are tons of open source projects out there who would love more hands on keyboards helping out. There is no excuse for not getting involved, even at a small degree.<br />
<br />
<h2>What's New in PHP 5.4</h2>Of all the talks, <a href="http://twitter.com/CalEvans">Cal Evans's</a> about the new features coming in PHP 5.4 was probably the least applicable to my current work, but was one of my favorites. Cal is an engaging speaker, and there is some really cool stuff coming up in PHP.<br />
<br />
The bits I was most interested in were improvements in Closures, Traits, and the <a href="http://php.net/manual/en/features.commandline.webserver.php">built-in HTTP server</a>. For those who don't know, in PHP 5.4, <a href="http://php.net/manual/en/functions.anonymous.php">closures</a> created in an object method will be able to reference $this from within the closure. That may not seem like a big deal, but if you're working with classes that build themselves or are factories for other objects, having the ability to reference the creating object is incredibly useful. The closure can access protected and private stuff from the instantiated class.<br />
<br />
The caveat is that $this will always reference the object in which the closure was created. It would be great to be able to bind $this to whatever object was invoking the closure, like you can with Javascript's .call() and .apply() functions.<br />
<br />
Lots has been written about <a href="http://php.net/manual/en/language.oop5.traits.php">Traits</a>. Of all the features in PHP 5.4, this is the one I'm most looking forward to. They will solve a lot of problems that currently can only be solved by having deeply nested class hierarchies. Combining traits with the ability to bind closures to an object paves the way for having completely dynamically created classes, which would be useful in many situations. It also gives more evidence for my theory that every programming language evolves until it can replicate Javascript.<br />
<br />
One other thing that Cal mentioned was the new <a href="http://php.net/manual/en/class.jsonserializable.php">JsonSerializable interface</a>. This is a great addition for anyone who deals with sending JSON responses in PHP. I only have one problem with it: there is no complementary JsonUnserializable interface that would allow an object to populate its properties from a JSON string. I don't watch the PHP internals list, so I don't know why it was decided not to include such obvious functionality. Does anyone have any insights?<br />
<br />
<h2>Refactoring and Other Small Animals</h2>After lunch, we got right into the second half of the day with <a href="http://twitter.com/mtabini">Marco Tabini</a> and a presentation on refactoring. The presentation centered around a piece of "legacy" code. He wrote a test to assert the existing behavior, then proceeded to follow the refactoring steps of <i>abstract</i>, <i>break (apart)</i> and <i>rename</i> to show how the code could be made more maintainable and testable.<br />
<br />
Marco's talk was to the point, with a great example, but it tilted more towards the theoretical side of <i>why</i> you should refactor. It also took a very purist look at what refactoring means (absolutely no changes to interface, even changes that would not affect calling code.) While I agree that this is the technical definition of refactoring, it's hard to start poking at a piece of code without wanting to improve its usage instead of just its implementation. I think there is a fuzzy line that developers cross often, at least mentally, between refactoring and improving the interface.<br />
<br />
At the after-party, some of my coworkers had a longer discussion with Marco where he expounded on some of the ideas he presented and gave some more practical advice for our own legacy application. I wasn't there, but I do appreciate speakers and experts taking the time to apply their own knowledge and experience to their audience's issues. So, thank you Marco.<br />
<br />
<h2>Jquery Mobile and PhoneGap</h2><a href="http://twitter.com/tpryan">Terry Ryan</a> was the representative from <a href="http://www.adobe.com">Adobe</a>, one of the conference sponsors. His talk began with a quick demo of building an app with <a href="http://jquerymobile.com/">jQuery Mobile</a>. I'll admit, I'm not a big fan of doing UI or front-end development, but even I could see how working with a library like that could make such development fun. It seems some of the kinks still need to be worked out, but it looks like a promising library and I hope to play with it some day soon.<br />
<br />
Following that was an overview of <a href="http://phonegap.com/">PhoneGap</a>, and more impressively, <a href="https://build.phonegap.com/">PhoneGap:Build</a>, where Terry tweaked his application, ran it through the cloud building service, and had the new app deployed to his phone in just a few minutes, all live. Impressive stuff!<br />
<br />
Then, Terry showed off some of the things Adobe has in the works with <a href="http://www.adobe.com/devnet/html5/articles/css-shaders.html">CSS shaders</a>. The effects he demoed were neat to look at, but I don't feel like there was anything there that <a href="http://www.chromeexperiments.com/webgl">WebGL</a> doesn't already cover.<br />
<br />
For a finale, Terry live demoed Adobe's <a href="http://www.adobe.com/products/proto.html">Proto</a> application, which allows tablet users to create UI wireframes and mock-ups by drawing with their fingers. Imagine a virtual whiteboard that takes your scratchings and turns them into divs, tables, text, headers and image frames live while you draw. Now imagine sharing that view with remote users, also live. Very cool stuff. <br />
<br />
<h2>REST Best Practices</h2><a href="http://twitter.com/CaseySoftware">Keith Casey's</a> presentation was an informative look into the philosophy and reasoning behind REST architecture, and was another one of my favorites. He talked about the myth of "strictly RESTful" and how the vast majority of APIs out there don't even come close (hint #1: if you're passing an authentication token or session ID on every request, your API is not RESTful.) I also liked his use of "-ilities": discoverability, readability, flexibility, maintainability, etc.<br />
<br />
He used <a href="http://www.twilio.com/">Twilio</a> as his example API, and showed how a good REST API should follow the <a href="http://en.wikipedia.org/wiki/Representational_state_transfer#Constraints">6 constraints of REST</a>: client-server, stateless (where most authentication schemes fail at being REST), cacheable, layered, uniform interface, and code-on-demand. The last constraint is the hardest one for me to get my head around, so I appreciated his use of GMail as an example of providing not just data, but the code that manipulates that data. I think that JSONP might also fall into that category, but I didn't get a chance to ask.<br />
<br />
Like with Marco's talk, I appreciated the theoretical nature of the topic, but I would have liked some more examples of real-world practicality trumping academic purity. It's always nice when a speaker acknowledges that sometimes you just need to get things done and points out which portions of a technology are more flexible than others and amenable to not following the letter of the law. Besides that, I thought it was a great look into why REST is the way it is.<br />
<br />
<h2>After-Party</h2>No development conference is complete without an after-party and no after-party is complete without free beer. I'd like to thank the folks at <a href="http://www.sugarcrm.com">SugarCRM</a> for sponsoring the after-party at <a href="http://www.tnnirishpub.com/">Tir na nOg</a>, one of my favorite pubs in downtown Raleigh.<br />
<br />
Besides free booze, I love to talk with speakers and other conference attendees in a less formal setting. It's great to get to know other developers at the top of their craft, and the people in the PHP community are some of the friendliest and easiest going. It's one of the reasons I love being a PHP developer so much. Keith, his coworker at Twilio <a href="http://twitter.com/devinrader">Devin Rader</a>, a few other people and I had a great conversation that shifted topics basically at the speed of thought. <a href="http://twitter.com/jmertic">John Mertic</a> from SugarCRM was there and answered some of my questions, and Beth explained the php|architect submission process in a way that didn't sound scary at all.<br />
<br />
To all the conference speakers, organizers and attendees, I want to say thank you for a fun and educational experience. Hopefully some of us will meet up again at <a href="http://tek12.phparch.com/">phptek 2012</a>! <br />Josh Adellhttp://www.blogger.com/profile/13146384900735324084noreply@blogger.com2tag:blogger.com,1999:blog-4037079221500014725.post-88007837025724755362011-11-05T14:28:00.002-04:002014-02-15T14:04:47.015-05:00Development Setup for Neo4j and PHP: Part 2<i>Update 2014-02-15: Using Neo4jPHP from a downloaded PHAR file has been deprecated. The preferred and supported way to install the library is via <a href="https://getcomposer.org/">Composer</a>. The examples below have been updated to reflect this. It has also been updated for the 2.0 release of Neo4j.</i><br />
<br />
This is Part 2 of a series on setting up a development environment for building projects using the graph database <a href="http://neo4j.org">Neo4j</a> and PHP. In <a href="http://blog.everymansoftware.com/2011/11/development-setup-for-neo4j-and-php.html">Part 1</a> of this series, we set up unit test and development databases. In this part, we'll build a skeleton project that includes unit tests, and a minimalistic user interface.<br />
<br />
All the files will live under a directory on our web server. In a real project, you'll probably want only the user interface files under the web server directory and your testing and library files somewhere more protected.<br />
<br />
Also, I won't be using any specific PHP framework. The principles in the code below should apply equally to any framework you decide to use.<br />
<br />
<h2>Create the Project and Testing Harness</h2>Create a project in a directory on your web server. For this project, mine is "/var/www/neo4play" on my local host. We'll also need a Neo4j client library. I recommend <a href="https://github.com/jadell/neo4jphp">Neo4jPHP</a> (disclaimer: I'm the author) and all the code samples in Part 2 use it.<br />
<pre class="prettyprint lang-sh">> cd /var/www/neo4play
> mkdir -p tests/unit
> mkdir lib
> echo '{"require":{"everyman/neo4jphp":"dev-master"}}' > composer.json
> composer install
> echo "<?php phpinfo(); ?>" > index.php
</pre>Test the setup by browsing to <a href="http://localhost/neo4play">http://localhost/neo4play</a>. You should see the output of `phpinfo`.<br />
<br />
Now we'll create a bootstrap file that we can include to do project-wide and environment specific setup. Call this file "bootstrap.php" in the root project directory.<br />
<pre class="prettyprint lang-php"><?php
require_once(__DIR__.'/vendor/autoload.php');
error_reporting(-1);
ini_set('display_errors', 1);
if (!defined('APPLICATION_ENV')) {
define('APPLICATION_ENV', 'development');
}
$host = 'localhost';
$port = (APPLICATION_ENV == 'development') ? 7474 : 7475;
$client = new Everyman\Neo4j\Client($host, $port);
</pre>The main point of this file at the moment is to differentiate between our development and testing environments, and set up our connection to the correct database. We do this by attaching the database client to the correct port based on an application constant.<br />
<br />
We'll use the bootstrap file to setup a different, unit testing specific bootstrap file. Create the following file as "tests/bootstap-test.php":<br />
<pre class="prettyprint lang-php"><?php
define('APPLICATION_ENV', 'testing');
require_once(__DIR__.'/../bootstrap.php');
// Clean the database
$query = new Everyman\Neo4j\Cypher\Query($client, 'MATCH n-[r]-m DELETE n, r, m');
$query->getResultSet();
</pre>The purpose of this file to to tell our application bootstrap that we are in the "testing" environment. Then it cleans out the database so that our tests run from a known state.<br />
<br />
Tell PHPUnit to use our test bootstrap with the following config file, called "tests/phpunit.xml":<br />
<pre class="prettyprint lang-xml"><phpunit colors="true" bootstrap="./bootstrap-test.php">
<testsuite name="Neo4j Play Test Results">
<directory>./unit</directory>
</testsuite>
</phpunit>
</pre>And because we're following <a href="http://en.wikipedia.org/wiki/Test-driven_development">TDD</a>, we'll create our first test file, "tests/unit/ActorTest.php":<br />
<pre class="prettyprint lang-php"><?php
class ActorTest extends PHPUnit_Framework_TestCase
{
public function testCreateActorAndRetrieveByName()
{
$actor = new Actor();
$actor->name = 'Test Guy '.rand();
Actor::save($actor);
$actorId = $actor->id;
self::assertNotNull($actorId);
$retrievedActor = Actor::getActorByName($actor->name);
self::assertInstanceOf('Actor', $retrievedActor);
self::assertEquals($actor->id, $retrievedActor->id);
self::assertEquals($actor->name, $retrievedActor->name);
}
public function testActorDoesNotExist()
{
$retrievedActor = Actor::getActorByName('Test Guy '.rand());
self::assertNull($retrievedActor);
}
}
</pre>So we know we want a domain object called "Actor" (apparently we're building some sort of movie application) and that Actors have names and ids. We also know we want to be able to look up an Actor by their name. If we can't find the Actor by name, we should get a `null` value back.<br />
<br />
Run the tests:<br />
<pre class="prettyprint lang-sh">> cd tests
> phpunit
</pre>Excellent, our tests failed! If you've been playing along, they probably failed because the "Actor" class isn't defined. Our next step is to start creating our domain objects.<br />
<br />
<h2>Defining the Application Domain</h2>So far, we only have one domain object, and a test that asserts its behavior. In order to make the test pass, we'll need to connect to the database, persist entities to it, and then query it for those entities.<br />
<br />
For persisting our entities, we'll need a way to get the client connection in our "Actor" class and any other domain object classes we define. To do this, we'll create an application registry/dependency-injection container/pattern-buzzword-of-the-month class. Put the following in the file "lib/Neo4Play.php":<br />
<pre class="prettyprint lang-php"><?php
class Neo4Play
{
protected static $client = null;
public static function client()
{
return self::$client;
}
public static function setClient(Everyman\Neo4j\Client $client)
{
self::$client = $client;
}
}
</pre>Now our domain objects will have access to the client connection through `Neo4Play::client()` when we persist them to the database. It's time to define our actor class, in the file "lib/Actor.php":<br />
<pre class="prettyprint lang-php"><?php
use Everyman\Neo4j\Node,
Everyman\Neo4j\Index;
class Actor
{
public $id = null;
public $name = '';
public static function save(Actor $actor)
{
}
public static function getActorByName($name)
{
}
}
</pre>Requiring our classes and setting up the client connection is part of the bootstrapping process of the application, so we'll need to add some thing to "bootstrap.php":<br />
<pre class="prettyprint lang-php"><?php
require_once(__DIR__.'/vendor/autoload.php');
require_once(__DIR__.'/lib/Neo4Play.php');
require_once(__DIR__.'/lib/Actor.php');
// ** set up error reporting, environment and connection... **//
Neo4Play::setClient($client);
</pre>We have a stub class for the domain object. The tests will still fail when we run them again, but at least all the classes should be found correctly.<br />
<br />
Let's start with finding an Actor by name. With our knowledge of graph databases, we know this will involve an index lookup, and that we will get a Node object in return. If the lookup returns no result, we'll get a `null`. If we do get a Node back, we'll want to hold on to it, for updating the Actor later.<br />
<br />
Modify the Actor class with the following contents:<br />
<pre class="prettyprint lang-php">class Actor
{
//
protected $node = null;
//
public static function getActorByName($name)
{
$actorIndex = new Index(Neo4Play::client(), Index::TypeNode, 'actors');
$node = $actorIndex->findOne('name', $name);
if (!$node) {
return null;
}
$actor = new Actor();
$actor->id = $node->getId();
$actor->name = $node->getProperty('name');
$actor->node = $node;
return $actor;
}
}
</pre>The main thing we're trying to accomplish here is keeping our domain classes as Plain-Old PHP Objects, that don't require any special class inheritance or interface, and that hide the underlying persistence layer from the outside world. <br />
<br />
The tests still fail. We'll finish up our Actor class by saving the Actor to the database.<br />
<pre class="prettyprint lang-php">class Actor
{
//
public static function save(Actor $actor)
{
if (!$actor->node) {
$actor->node = new Node(Neo4Play::client());
}
$actor->node->setProperty('name', $actor->name);
$actor->node->save();
$actor->id = $actor->node->getId();
$actorIndex = new Index(Neo4Play::client(), Index::TypeNode, 'actors');
$actorIndex->add($actor->node, 'name', $actor->name);
}
//
}
</pre>Run the tests again. If you see all green, then everything is working properly. To double check, browse to the testing instance webadmin panel <a href="http://localhost:7475/webadmin/#">http://localhost:7475/webadmin/#</a>. You should see 2 nodes and 1 property (Why 2 nodes? Because there is a node 0 -- the reference node -- that is not deleted when the database is cleaned out.)<br />
<br />
<h2>Build Something Useful</h2>It's time to start tacking on some user functionality to our application. Thanks to our work on the unit tests, we can create actors in the database and find them again via an exact name match. Let's expose that functionality.<br />
<br />
Change the contents of "index.php" to the following:<br />
<pre class="prettyprint lang-php"><?php
require_once('bootstrap.php');
if (!empty($_POST['actorName'])) {
$actor = new Actor();
$actor->name = $_POST['actorName'];
Actor::save($actor);
} else if (!empty($_GET['actorName'])) {
$actor = Actor::getActorByName($_GET['actorName']);
}
?>
<form action="" method="POST">
Add Actor Name: <input type="text" name="actorName" />
<input type="submit" value="Add" />
</form>
<form action="" method="GET">
Find Actor Name: <input type="text" name="actorName" />
<input type="submit" value="Search" />
</form>
<?php if (!empty($actor)) : ?>
Name: <?php echo $actor->name; ?><br />
Id: <?php echo $actor->id; ?><br />
<?php elseif (!empty($_GET['actorName'])) : ?>
No actor found by the name of "<?php echo $_GET['actorName']; ?>"<br />
<?php endif; ?>
</pre>Browse to your index file. Mine is at <a href="http://localhost/neo4play/index.php">http://localhost/neo4play/index.php</a>.<br />
<br />
You should see the page you just created. Enter a name in the "Add Actor Name" box and click the "Add" button. If everything went according to plan, you should see the actor name and the id assigned to the actor by the database.<br />
<br />
Try finding that actor using the search box. Note the actor's id.<br />
<br />
Browse to <a href="http://localhost:7474/webadmin/#">http://localhost:7474/webadmin/#</a> and click the "Data browser" tab. Enter the actor id in the text box at the top. The node you created when you added the actor should show up.<br />
<br />
The interesting thing is that our actual application doesn't know anything about how the Actors are stored. Nothing in "index.php" references graphs or nodes or indexes. This means that, in theory, we could swap out the persistence layer for a SQL databases later, or MongoDB, or anything else, and nothing in our application would have to change. If we started with a SQL database, we could easily transition to a graph database.<br />
<br />
<h2>Explore the Wonderful World of Graphs</h2>Your development environment is now set up, and your application is bootstrapped. There's a lot more to add to this application, including creating movies, and linking actors and movies together. Maybe you'll want to add a social aspect, with movie recommendations. Graph databases are powerful tools that enable such functionality to be added easily.<br />
<br />
Go ahead and explore the rest of the Neo4jPHP library (<a href="http://github.com/jadell/Neo4jPHP/wiki">wiki</a> and <a href="http://jadell.github.com/Neo4jPHP">API</a>). Also, be sure to checkout the <a href="http://docs.neo4j.org">Neo4j documentation</a>, especially the sections about the REST API, Cypher and Gremlin (two powerful graph querying and processing languages.)<br />
<br />
All the code for this sample application is available as a gist: <a href="http://gist.github.com/1341833">http://gist.github.com/1341833</a>.<br />
<br />
Happy graphing!Josh Adellhttp://www.blogger.com/profile/13146384900735324084noreply@blogger.com23tag:blogger.com,1999:blog-4037079221500014725.post-91772503919618109532011-11-05T02:47:00.000-04:002014-02-15T14:14:15.713-05:00Development Setup for Neo4j and PHP: Part 1<i>Update 2014-02-15: Using Neo4jPHP from a downloaded PHAR file has been deprecated. The preferred and supported way to install the library is via Composer. The examples below have been updated to reflect this. It has also been updated for the 2.0 release of Neo4j.</i><br />
<br />
I would really love to see more of my fellow PHP developers talking about, playing with, and building awesome applications on top of graph databases. They really are a powerful storage solution that fits well into a wide variety of domains.<br />
<br />
In this two part series, I'll detail how to set up a development environment for building a project with a graph database (specifically <a href="http://neo4j.org">Neo4j</a>). Part 1 will show how to set up the development and unit testing databases. In <a href="http://blog.everymansoftware.com/2011/11/development-setup-for-neo4j-and-php_05.html">Part 2</a>, we'll create a basic application that talks to the database, including unit tests.<br />
<br />
All the steps below were performed on Ubuntu 10.10 Maverick, but should be easy to translate to any other OS.<br />
<br />
<h2>Grab the Components</h2>There are a few libraries and software tools needed for the project. First off, we'll need to install our programming environment, specifically PHP, PHPUnit and Java. How to do this is dependent on your operating system. My setup is PHP 5.3, PHPUnit 3.6 and Sun Java 6.<br />
<br />
Next, we need to get Neo4j. Download the latest tarball from <a href="http://neo4j.org/download">http://neo4j.org/download</a>. I usually go with the latest milestone release (2.0 at this time.)<br />
<pre class="prettyprint lang-sh">> cd ~/Downloads
> wget http://dist.neo4j.org/neo4j-community-2.0.1-unix.tar.gz
</pre><br />
<h2>Set Up the Development Instance</h2>It is possible to put test and development data in the same database instance, but I prefer a two database solution, even when using a SQL database. I do this because 1) I don't have to worry about accidentally blowing away any development data I care about, and 2) I don't have to worry about queries and traversals in my unit tests accidentally pulling in development data, and vice versa.<br />
<br />
Unlike SQL or many NOSQL database servers, Neo4j cannot have more than one database in a single instance. The only way to get two databases is to run two separate instances on two separate ports or hosts. The instructions below describe how to set up our development and unit testing databases on the same host but listening on different ports.<br />
<br />
First, create a point to hold both neo4j instances, and unpack the development instance:<br />
<pre class="prettyprint lang-sh">> mkdir -p ~/neo4j
> cd ~/neo4j
> tar -xvzf ~/Downloads/neo4j-community-2.0.1-unix.tar.gz
> mv neo4j-community-2.0.1 dev
</pre>Next, configure the server by editing the file "~/neo4j/dev/conf/neo4j-server.properties". Make sure that the server is on port 7474<br />
<pre class="prettyprint lang-sh">org.neo4j.server.webserver.port=7474
</pre>If you will be accessing the database instance from a host other than localhost, uncomment the following line:<br />
<pre class="prettyprint lang-sh">org.neo4j.server.webserver.address=0.0.0.0
</pre>Save the config and exit. Now it's time to start the instance.<br />
<pre class="prettyprint lang-sh">> ~/neo4j/dev/bin/neo4j start
</pre>You should be able to browse to the Neo4j Browser panel: <a href="http://localhost:7474">http://localhost:7474</a><br />
<br />
<h2>Set Up the Testing Instance</h2>Unpack the test instance:<br />
<pre class="prettyprint lang-sh">> cd ~/neo4j
> tar -xvzf ~/Downloads/neo4j-community-2.0.1-unix.tar.gz
> mv neo4j-community-2.0.1 test
</pre>Next, configure the server by editing the file "~/neo4j/test/conf/neo4j-server.properties". Make sure that the server is on port 7475 (note that this is a <strong>different port</strong> from the development instance!)<br />
<pre class="prettyprint lang-sh">org.neo4j.server.webserver.port=7475
</pre>If you will be accessing the database instance from a host other than localhost, uncomment the following line:<br />
<pre class="prettyprint lang-sh">org.neo4j.server.webserver.address=0.0.0.0
</pre>We need one more change to differentiate the testing from the development instance. Edit the file "~/neo4j/test/conf/neo4j-wrapper.properties" and change the instance name line:<br />
<pre class="prettyprint lang-sh">wrapper.name=neo4j-test
</pre>Save and exit, then start the instance.<br />
<pre class="prettyprint lang-sh">> ~/neo4j/test/bin/neo4j start
</pre>You should be able to browse to the Neo4j Browser panel: <a href="http://localhost:7475">http://localhost:7475</a><br />
<br />
Congratulations! Everything is set up and ready for us to build our application. In <a href="http://blog.everymansoftware.com/2011/11/development-setup-for-neo4j-and-php_05.html">Part 2</a> of this series, I'll talk about setting up unit tests and building a basic application using a graph database as a backend. Josh Adellhttp://www.blogger.com/profile/13146384900735324084noreply@blogger.com22tag:blogger.com,1999:blog-4037079221500014725.post-90854886541451499922011-10-04T23:10:00.003-04:002011-10-04T23:12:59.522-04:00A Humbling ReferenceSometimes I have an experience that reminds me of how near-sighted I can get when I'm head-down in a problem.<br />
<br />
What do you think the output of the following code will be (no cheating by running it first!):<br />
<pre class="prettyprint lang-php"><?php
$a = array(array("result" => "a"));
for ($i=0; $i < 2; $i++) {
foreach ($a as $j => $v) {
$v["result"] .= "a";
echo $v["result"] ."\n";
}
}
foreach ($a as $k => $v) echo $v["result"] ."\n";
</pre>If the goal is to add the character "a" twice onto each result element of each sub-array, at first glance this seems like a perfectly reasonable way to go about it.<br />
<br />
But hold up! Each time through, we seem to have only added one "a" onto the result. And in the second foreach loop, it seems like we didn't modify the value at all! What gives?<br />
<br />
Well, it might be obvious to most of you, but when it was costing me an hour and a half of my work day it was buried in a nest of process forking and stream handling, part of which looked something like this:<br />
<pre class="prettyprint lang-php">while ($currentExecution) {
foreach ($currentExecution as $i => $handle) {
$handle['output'] .= stream_get_contents($handle['outstream']);
$status = proc_get_status($handle['process']);
if (!$status['running']) {
fclose($handle['outstream']);
proc_terminate($handle['process']);
echo $handle['output']."\n";
unset($currentExecution[$i]);
}
}
}
</pre>Same concept as before: I have an array of process handles (opened with <a href="http://php.net/manual/en/function.proc-open.php">proc_open</a>) and I'm looping through, gathering the output of each process as it is generated, until the process is finished running, at which point I close the process and display the output. The symptom was that, sometimes, I would only see the end of the output and not the beginning.<br />
<br />
The times when this happened were the times where a process handle went through the outer while loop more than once, the same as in the contrived example above. The reason for this is because when you are working with a value inside a foreach loop, you are actually working with a <em>copy</em> of that value (technically a copy-on-write). So modifications made to the value do not persist through multiple passes through the while loop.<br />
<br />
As it turns out, the solution here is quite simple. Actually, there are several solutions. Here's the easiest one, applied to the original example:<br />
<pre class="prettyprint lang-php"><?php
$a = array(array("result" => "a"));
for ($i=0; $i < 2; $i++) {
foreach ($a as $j => &$v) {
$v["result"] .= "a";
echo $v["result"] ."\n";
}
}
foreach ($a as $k => $v) echo $v["result"] ."\n";
</pre>The difference is subtle: it's the "&" prepended to the $v in the foreach loop. This tells the loop to use a <em>reference</em> to the value instead of a copy. Here's another possible solution:<br />
<pre class="prettyprint lang-php"><?php
$a = array(array("result" => "a"));
for ($i=0; $i < 2; $i++) {
foreach ($a as $j => $v) {
$a[$j]["result"] .= "a";
echo $a[$j]["result"] ."\n";
}
}
foreach ($a as $k => $v) echo $v["result"] ."\n";
</pre>In this case, we're not even using the variable $v. We're using the array index to refer to the value inside of the array directly. Because we're modifying the actual array value and not the copy, the changes persist through the loop.<br />
<br />
The final solution (and the one I ended up going with) is to make the value an object instead of an array:<br />
<pre class="prettyprint lang-php"><?php
$obj->result = "a";
$a = array($obj);
for ($i=0; $i < 2; $i++) {
foreach ($a as $j => $v) {
$v->result .= "a";
echo $v->result ."\n";
}
}
foreach ($a as $k => $v) echo $v->result ."\n";
</pre>Objects in PHP are always passed around by reference, including in foreach loops. So modifying the object's property inside the loop modifies the actual object, not a copy.<br />
<br />
The most annoying thing about this whole experience? I had solved this problem for someone else a few weeks ago, telling them what the issue was just from hearing them describe the situation and without even seeing their code run. So the deeper lesson here is this: when you've been staring at code for so long that it's making you cross-eyed and frustrated, walk away. And most importantly, get someone else who hasn't been working on it to look it over.<br />
<br />
...something something forest...something something trees... <br />
<br />
Sample code available at <a href="http://gist.github.com/1263522">http://gist.github.com/1263522</a><br />
<br />
Josh Adellhttp://www.blogger.com/profile/13146384900735324084noreply@blogger.com2tag:blogger.com,1999:blog-4037079221500014725.post-61764892155529752142011-09-23T20:33:00.001-04:002011-09-25T09:21:14.998-04:00Phar Flung PhingOne of the cooler features of PHP 5.3 is the ability to package up a set of PHP class files and scripts into a single archive, known as a <a href="http://php.net/manual/en/book.phar.php">PHAR</a> ("PHp ARchive"). PHAR files are pretty much equivalent to Java's JAR files: they allow you to distribute an entire library or application as a single file. I decided to see how easy it would be to wrap up <a href="http://github.com/jadell/Neo4jPHP">Neo4jPHP</a> in a PHAR for distribution.<br />
<br />
Surprisingly, there isn't much information out there on creating PHAR files outside of the PHP manual and a <a href="http://www.ibm.com/developerworks/opensource/library/os-php-5.3new4/index.html">few</a> older <a href="http://blog.calevans.com/2009/07/19/lessons-in-phar/">posts</a>. They deal mainly with creating a PHAR with a hand-written script specific to the project being packaged, using PHP's PHAR classes and methods.<br />
<br />
Since I also started playing with <a href="http://www.phing.info/">Phing</a> recently, I decided to see if I could incorporate packaging a project as a PHAR into my build system. It turns out, it's pretty easy, given that Phing has a built-in PharPackage task.<br />
<br />
All steps below were performed on Ubuntu 10.10 Maverick<br />
<br />
<h3>Step 1 - Get Phing</h3>The Phing docs are pretty explicit and easy to follow. If you have PEAR installed, simply do:<br />
<pre class="prettyprint lang-sh">> sudo pear channel-discover pear.phing.info
> sudo pear install phing/phing
</pre>If all goes well, you should the location of the phing command line utility when you run<br />
<pre class="prettyprint lang-sh">> which phing
</pre><br />
<h3>Step 2 - Allow PHP to create PHARs</h3>This is one that most of the posts I found on creating PHARs fail to mention. In the default installation of PHP, PHARs are read-only (this prevents a nasty security hole, whereby any PHP script on your machine has the ability to modify other PHARs.) In order to prevent this, the php.ini setting "phar.readonly" must be turned off. I did this by creating a new file in my PHP conf.d directory:<br />
<pre class="prettyprint lang-sh">> sudo sh -c "echo 'phar.readonly=0' > /etc/php5/conf.d/phar.ini"
</pre><br />
<h3>Step 3 - Create a stub</h3>In a PHAR, the stub file is run when the PHAR is `include`d or `require`d in a script (or when the PHAR is invoked from the command line.) The purpose of the stub is to do any setup necessary for the library contained in the PHAR. Since I'm packaging a library, the stub I created doesn't do anything more than set up autoloading of my library's classes.<br />
<br />
Save the following file as "stub.php" in the root of your project:<br />
<pre class="prettyprint lang-php"><?php
Phar::mapPhar('myproject.phar');
spl_autoload_register(function ($className) {
$libPath = 'phar://myproject.phar/lib/';
$classFile = str_replace('\\',DIRECTORY_SEPARATOR,$className).'.php';
$classPath = $libPath.$classFile;
if (file_exists($classPath)) {
require($classPath);
}
});
__HALT_COMPILER();
</pre>The "__HALT_COMPILER();" line <strong>must</strong> be the last thing in the file. The call to `Phar::mapPhar()` registers the PHAR with PHP and allows any of the files inside to be found via the "phar://" stream wrapper. Any files or directories contained in the PHAR can be referenced as "phar://myproject.phar/path/inside/phar/MyClass.php". Since all my project files are namespaced, autoloading is a matter of translating the class name into a path inside the PHAR.<br />
<br />
<h3>Step 4 - Create a build file</h3>Phing's documentation is pretty extensive, so I won't go into to much detail about what each piece of the build file means. The important part is the <a href="http://www.phing.info/docs/guide/stable/chapters/appendixes/AppendixC-OptionalTasks.html#PharPackageTask">PharPackage</a> task in the "package" target.<br />
<br />
Save the following file as "build.xml" in the root of your project:<br />
<pre class="prettyprint lang-xml"><?xml version="1.0" encoding="UTF-8"?>
<project name="MyProject" default="package">
<!-- Target: build -->
<target name="build">
<delete dir="./build" />
<mkdir dir="./build" />
<copy todir="./build">
<fileset dir=".">
<include name="lib/" />
</fileset>
</copy>
</target>
<!-- Target: package -->
<target name="package" depends="build">
<delete file="./myproject.phar" />
<pharpackage
destfile="./myproject.phar"
basedir="./build"
compression="gzip"
stub="./stub.php"
signature="sha1">
<fileset dir="./build">
<include name="**/**" />
</fileset>
<metadata>
<element name="version" value="1.2.3" />
<element name="authors">
<element name="Joe Developer">
<element name="email" value="joe.developer@example.com" />
</element>
</element>
</metadata>
</pharpackage>
</target>
</project>
</pre>The important part is the "package" target, the default target of this build file. It depends on the "build" target, whose only purpose is to copy out all the files that will be packaged in the PHAR into a separate "build" directory. This strips out any testing, example or other non-library files. Any previously created package is deleted.<br />
<br />
The main work of packaging is done via the "pharpackage" task. In this case, I'm specifying that the output package should be called "myproject.phar" and be put in the same directory as the build file, the PHAR should be compressed with "gzip", the stub to load when requiring the file is at "./stub.php" and the PHAR should be signed with a SHA1 hash.<br />
<br />
Metadata appears to be optional, but Phing throws up a nasty looking warning if it's not there (it tries to convert an empty value to an array.) Nested metadata is turned into a PHP array and serialized.<br />
<br />
<h3>Step 5 - Phling your PHAR</h3>Run the following command in the directory with your build file:<br />
<pre class="prettyprint lang-sh">> phing
# or, if the "package" task is not the default
> phing package
</pre>If all went well, there should be a file called "myproject.phar" in the root of your project. You can test it out by running this test script:<br />
<pre class="prettyprint lang-php"><?php
require("myproject.phar");
$thing = new Class\In\My\Project();
</pre>If PHP doesn't complain about a missing class, the PHAR set up correctly and autoloading is working.<br />
<br />
The Neo4jPHP PHAR is available at <a href="http://github.com/downloads/jadell/Neo4jPHP/neo4jphp.phar">http://github.com/downloads/jadell/Neo4jPHP/neo4jphp.phar</a>.<br />
<br />
Josh Adellhttp://www.blogger.com/profile/13146384900735324084noreply@blogger.com5tag:blogger.com,1999:blog-4037079221500014725.post-63099271872062931102011-08-25T10:26:00.003-04:002011-08-25T13:30:56.866-04:00Getting Neo4jPHP working with CodeIgniter 2<i>This post is based heavily off of <a href="http://wildlyinaccurate.com/integrating-doctrine-2-with-codeigniter-2/">Integrating Doctrine 2 with CodeIgniter 2</a>. I haven't tested it myself, so please use it as a starting point and let me know if I should update anything.</i><br />
<br />
Here are the steps to get <a href="http://github.com/jadell/Neo4jPHP">Neo4jPHP</a> set up as a CodeIgniter 2 library:<br />
<ol><li>Copy the 'Everyman' folder to application/libraries</li>
<li>Create a file called Everyman.php in application/libraries</li>
<li>Copy the following code into Everyman.php:<br />
<pre class="prettyprint lang-php"><?php
class Everyman
{
public function __construct()
{
spl_autoload_register(array($this,'autoload'));
}
public function autoload($sClass)
{
$sLibPath = __DIR__.DIRECTORY_SEPARATOR;
$sClassFile = str_replace('\\',DIRECTORY_SEPARATOR,$sClass).'.php';
$sClassPath = $sLibPath.$sClassFile;
if (file_exists($sClassPath)) {
require($sClassPath);
}
}
}
</pre></li>
<li>Add the following line to application/config/autoload.php:<br />
<pre class="prettyprint lang-php">$autoload['libraries'] = array('everyman');
</pre></li>
</ol><br />
This is actually a generic autoloader that will attempt to autoload any namespaced class under application/library where the class name matches the file path.<br />
<br />
Josh Adellhttp://www.blogger.com/profile/13146384900735324084noreply@blogger.com18tag:blogger.com,1999:blog-4037079221500014725.post-87752430805206090212011-07-23T16:06:00.001-04:002011-07-23T21:28:05.307-04:00Performance of Graph vs. Relational DatabasesA few weeks ago, Emil Eifrem, CEO of <a href="http://neotechnology.com/">Neo Technology</a> gave a webinar <a href="http://www.youtube.com/watch?v=UodTzseLh04">introduction to graph databases</a>. I watched it as a lead up to my own presentation on graph databases and Neo4j.<br />
<br />
Right around the 54 minute mark, Emil talks about a very interesting experiment showing the performance difference between a relational database and a graph database for a certain type of problem called "arbitrary path query", specifically, given 1,000 users with an average of 50 "friend" relationships each, determine if one person is connected to another in 4 or fewer hops.<br />
<br />
Against a popular open-source relational database, the query took around 2,000 ms. For a graph database, the same determination took 2 ms. So the graph database was 1,000 times faster for this particular use case.<br />
<br />
Not satisfied with that, they then decided to run the same experiment with 1,000,000 users. The graph database took 2 ms. They stopped the relational database after several days of waiting for results.<br />
<br />
I showed this clip at my presentation to a few people who stuck around afterwards, and we tried to figure out why the graph database had the same performance with 1,000 times the data, while the relational database became unusably slow. The answer has to do with the way in which each type of database searches information.<br />
<br />
Relational databases search all of the data looking for anything that meets the search criteria. The larger the set of data, the longer it takes to find matches, because the database has to examine everything in the collection.<br />
<br />
Here's an example: let's assume there is a table with "friend" relationships:<br />
<pre class="prettyprint lang-sql">> SELECT * FROM friends;
+-------------+--------------+
| user_id | friend_id |
+-------------+--------------+
| 1 | 2 |
| 1 | 3 |
| 1 | 4 |
| 2 | 5 |
| 2 | 6 |
| 2 | 7 |
| 3 | 8 |
| 3 | 9 |
| 3 | 10 |
+-------------+--------------+
</pre>In order to see if user "9" is connected to user "2", the database has to find all the friends of user "9", and see if user "2" is in that list. If not, find all of their friends, and then see if user "2" is in that list. The database has to scan the entire table each time. This means that if you double the number of rows in the table, you've doubled the amount of data to search, and thus doubled the amount of time it takes to find what you are looking for. (Even with indexing, it still has to find the values in the index tree, which involves traversing that tree. The index tree grows larger with each new record, meaning the time it takes to traverse grows larger as well. And for each search, you always start at the root of the tree.)<br />
<br />
Each new batch of friends to look at requires an entirely new scan of the table/index. So more records leads to more search time.<br />
<br />
Conversely, a graph database looks only at records that are directly connected to other records. If it is given a limit on how many "hops" it is allowed to make, it can ignore everything more than that number of hops away.<br />
<div class="separator" style="clear: both; text-align: center;"></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiyVpsNq0Kxt8suLB17xyFfliePJt1i1ZRKNhK69OjG_nl3k0qrqVcmaXUyQp6hZx6LhYFoicF_rYTTSGMnt9OyRxJ7wLdThx948Ec_BGD3GgMYRkmGCIGXF5GX2PbzAf-RTg5Y06-0Xo-9/s1600/fofgraph+%25281%2529.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="289" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiyVpsNq0Kxt8suLB17xyFfliePJt1i1ZRKNhK69OjG_nl3k0qrqVcmaXUyQp6hZx6LhYFoicF_rYTTSGMnt9OyRxJ7wLdThx948Ec_BGD3GgMYRkmGCIGXF5GX2PbzAf-RTg5Y06-0Xo-9/s320/fofgraph+%25281%2529.png" width="320" /></a></div>Only the blue records are ever seen during the search. And since a graph traversal "remembers" where it is at any time, it never has to start from the beginning, only from its last known position.<br />
<br />
You could add another ring of records around the outside, or add a thousand more rings of records outside, but the search would never need to look at them because they are more steps away than the limit. The only way to increase the number of records searched (and thereby decrease performance) is to add more records within the 2-step limit, or add more relationships between the existing records.<br />
<br />
So the reason why having 1,000 vs. 1,000,000 records causes such a stark difference between a relational and a graph database is that relational database performance decreases in relation to the number of records in the table, while graph database performance decreases in relation to the number of connections between the records.Josh Adellhttp://www.blogger.com/profile/13146384900735324084noreply@blogger.com6tag:blogger.com,1999:blog-4037079221500014725.post-29117865783811991532011-07-19T22:58:00.001-04:002011-07-19T23:00:32.755-04:00TriPUG Meetup slidesHere are the slides for my presentation on graph databases at the <a href="http://www.meetup.com/mysql-144/">TriPUG meetup</a> tonight: <a href="http://www.slideshare.net/JoshAdell/graphing-databases">http://www.slideshare.net/JoshAdell/graphing-databases</a><br />
<br />
Overall, I think it was pretty well received. I didn't mean for it to last an hour and a half, but there it is. There were a lot of new folks there; I'm encouraged at the enthusiasm of the developer community in the Triangle region. Thanks for having me!Josh Adellhttp://www.blogger.com/profile/13146384900735324084noreply@blogger.com0