Mirror Drupal 7 nodes on a second Drupal 7 site

This should be dead simple, but it's not... yet.

A survey of the options

Data export import module, currently in bailey86's sandbox
http://drupal.org/sandbox/bailey86/1278830
and described rather extensively here:
http://groups.drupal.org/node/195178
"The purpose of this module is to provide a way to move data between Drupal instances." However it does it with a method that does not lend itself to automation: it "exports nodes and taxonomy terms to data files - which can then be imported into other Drupal instances"

http://drupal.org/project/syndeploy bills itself as a pull version of Deploy (and actually we would prefer push), but it is in any case Drupal 6 not Drupal 7.
http://london2011.drupal.org/conference/sessions/content-staging-and-deployments-drupal-7
http://drupal.org/project/deploy - Deployment module itself is not meant for partial content and the Drupal 7 version is in dev.

But it is used in Drupal 7. http://london2011.drupal.org/conference/sessions/content-staging-and-deployments-drupal-7 And for precisely our use case: publishers wants to deploy content from one site to a network of sub sites, or similar.

Ha! I was quoted in that presentation:

The Best of All Possible Worlds?
"If we can flag what is content (meaning from a workflow
perspective controlled on the live site) and what is configuration
(meaning for the ordinary workflow that the code will expect to
override the on-site configuration[...]) on a per-bundle basis i
think we get the best of all possible worlds :-)"
- Benjamin Melançon
http://drupal.org/node/938368#comment-4286326

Key takeaway (from the presentation, not my cameo): Deployment module for Drupal 7 uses Services to do the actual deploy. So perhaps it in its own way is the best of all possible worlds! (I've been wary of using Deploy seeing as how its creator, Greg Dunlap, fled it for the official Configuration Management initiative, but it is (dev or not) in the capable hands of Katherine Bailey and Dick Olsson.

This fellow asks about "Content Syndication using Feeds. How do I make it secure?" but lists an interesting module base as what he is looking at:
Feeds Atom, Views Atom, and Push_Hub
http://groups.drupal.org/node/169904
no testing or answers there though.

http://drupal.org/project/push_hub however only uses RSS or Atom. Rich enough information?

"This synchronization of Drupal nodes using Pubsubhubbub is getting more and more mature. It’s still not using RDF, but it’s now secure and will allow you to synchronize nodes that do not have the exact same structure. With Drupal 7 and its support for RDF, this way of synchronizing Drupal nodes could be extended to entities and become a simple and efficient way of synchronizing various Drupal platforms."
- http://www.viguierjust.com/en/2011/02/06/drupal-6-complex-node-synchronization-with-pubsubhubbub/

There's what looks like a good approach - http://www.viguierjust.com/en/2011/02/06/drupal-6-complex-node-synchronization-with-pubsubhubbub/ - for Pubsubhubbub that relies on Feeds Atom - http://drupal.org/project/feeds_atom - but that module is Drupal 6 only, with upgrade expected in a couple months.

Migrating some content to a new Drupal site from an old Drupal site mentioned:
Comparison of Content and User Import and Export Modules
http://groups.drupal.org/node/21338

How to use Migrate module for nightly import
http://groups.drupal.org/node/111199
he's using it successfully.

http://drupal.org/project/migrate
and
http://drupal.org/project/feeds
are variously recommended

I'm really leaning toward neither of them and Services based instead.

Then there's Using drush to synchronize and deploy sites. http://drupal.org/node/670460

Should report back answers to:
Drupal content sync http://drupal.org/node/1039530

How Drupal 8 might fix everything, from a Drupal critic:
http://davidherron.com/node/7550

The Services Way

http://drupal.org/project/services

Drupal 6 how-tos:
http://www.delawarewebdesigner.com/tech-stuff/sync-drupal-content-using-services-and-xmlrpc.htm
http://www.delawarewebdesigner.com/how-tos/sync-files-in-drupal-using-services-and-xmlrpc.htm

Aside: Services and Rules for Drupal-to-Drupal data sync is tantalizing, and doable, but probably not quite there. Here's a start on Services and Rules, http://drupal.org/project/wsclient

Drupal 7 straight services, XML-RPC style:
http://awebfactory.com.ar/node/297

On the target site, enable Services and REST Server.

Disabled Overlay, just because i can't deal with using it.

drush dl deploy uuid

deploy 7.x-2.x-dev  -  2011-Nov-21  -  Supported, Development
uuid (7.x-1.0-alpha2)

To be sort of future proof i put the library in libraries and symlinked into the place in the module where the module currently looks.
ln -s ~/public_html/weddev/sites/all/libraries/spyc-0.5/spyc.php ~/public_html/weddev/sites/all/modules/contrib/services/servers/rest_server/lib/spyc.php

On the recipient site

admin/structure/services/add

Add a new endpoint

Name * Business listing (The unique ID for this endpoint.)
Server * REST (Select a the server that should be used to handle requests to this endpoint.)
Path to endpoint * services/biz
Debug mode enabled [checked for test] (Useful for developers. Do not enable on production environments)
Authentication [left unchecked for test] (Session authentication)

http://weddev.he236.vps.webenabled.net/services/biz

On the source site

Add the endpoint to the source site's listing

admin/structure/deploy/endpoints/add

Title Wedding site

Can skip the description.

For Authenticator, select "Session authentication Performs session authentication on the endpoint. Works well when the endpoint happens to be a Drupal site using Services module and session authentication." ... because that's the only one that currently works.

For Service, select "REST JSON Deploys over a REST service that can receive JSON data. Works well when the endpoint happens to be a Drupal sites using Services module with REST server." ... as it's the only option anyway.

On the next, Authenticator, page, put in the username and password of a user on the target site that has sufficient privileges.

Press Continue, and on the final, Service, page, put in the URL of your destination site plus the path to the endpoint you configured there. Locally that was http://weddev.localhost/services/biz

Press Finish.

Make a plan

Now go to the Plan tab and add a new plan:

admin/structure/deploy/plans/add

Listing1

The description might be more useful here.

For Aggregator, select "Views aggregator Aggregates content to a deployment plan by executing a user defined view."

For Processor, recommend to use "Queue API All entities are queued for deployment with the Queue API. Works best with large deployments. Can be used with a separate queue worker to achive very fast and parallel deployments." for production but can try Memory processor for testing small amounts of content.

For Endpoint, select the one you just made.

Press Continue.

Select a View to use, continue, and then Finish.

Problems since dealt with

he236!w_bizdev:~/public_html/bizdev/sites/all/modules$ drush cron
WD deploy: Request response error: 403 Forbidden                              [error]
WD deploy: Request response error: 403 Forbidden                              [error]
WD deploy: Request response error: 403 Forbidden                              [error]
WD deploy: Request response error: 403 Forbidden                              [error]
WD deploy: Request response error: 403 Forbidden                              [error]
WD deploy: Request response error: 403 Forbidden                              [error]
WD deploy: Request response error: 403 Forbidden                              [error]
WD deploy: Request response error: 403 Forbidden                              [error]
Cron run successfully.                                                        [success]

We have different ideas of successful.

Gah, horrible UI. It's up at the top:

"Basics » Authenticator » Service" where those double arrows don't signify breadcrumb hierarchy like elsewhere in Drupal

Here's one big thing i was missing, on the receiving site:

drush dl uuid
Project uuid (7.x-1.0-alpha2) downloaded to [success]
/home/clients/websites/w_weddev/public_html/weddev/sites/all/modules/contrib/uuid.
Project uuid contains 3 modules: uuid_services_example, uuid_services, uuid.

Did not enable the example, a feature.

Went to the existing service we created and edited resources, admin/structure/services/list/biz/resources and enabled uuid_entity there.

http://weddev.he236.vps.webenabled.net/biz/uuid_entity/05a5d836-7a54-0594-1922-9c530a41417e.json

"You don't have permission to access /biz/uuid_entity/05a5d836-7a54-0594-1922-9c530a41417e.json on this server."

See also:

• Victor Kane, leveraging the services module: http://awebfactory.com.ar/node/297
• TODO, test and RTBC - use libraries for Services: http://drupal.org/node/1325572