image uploads and version control?

[lawpoop]

I'm developing a php website that I have under subversion version
control. I'm working on an image upload functionality, and in the
middle of it I realized that any files that a user uploads will not be
under version control, and if I checkout or export the site from
version control, and deploy it, it won't bring any of the uploaded
files with it.

I'm looking at php subversion functions, but the manual says that
they're experimental, and my webhost hasn't deployed them.

What's a workaround for this? I'm thinking that I'll have to have a
parallel site for image hosting. For example, I'll have
images.website. com, hosted under a separate filesystem directory, and
then in the actual website that a user would peruse, any img src would
reference images.website. com . When I accept my uploads, I'll do my
filesystem copy to the images filesystem link.

Thoughts?

[lawpoop]

On Oct 13, 12:38*pm, Michael Fesser <[e-mail] wrote:

.oO(lawp...@gma il.com)

>

Well, I can do all that in one fell swoop by creating a symbolic link.
I don't have to worry about whether or not I've deleted anything in
version control that I then need to remove in the production; it's
already done for me. If I had to delete files what were already
deleted in version control, then in my mind, I'm doing manual labor
that the computer should be doing anyway.

That's what synchronizing is for. The computer can easily check which
files have been modified or deleted from the working copy and upload all
required changes to the production server. There are various tools
available for this task.

I suppose there are tools for this. Care to give me a sales pitch on
any of them? Right now an svn export does everything I want, with no
risks of the downsides of copying or synchronizing.
And the deployment tool is built right into the version control
system. Why do I need an extra tool here?

>

Also, by using symbolic links, I can deploy, test, and fallback to the
old directory by re-linking very easily if I need to.

This should happen on a testing server. You hardly need multiple
versions on the production server.

That would always work if the development and production environments
were identical, but in my experience, they're not. Code that passes a
test in the development environment might still have problems when you
roll it out in production. And then that happens, you'll want to roll
it back *really really badly*. :)

The situation I've been involved in before was where we made a change
to the database structure in development which we didn't replicate in
the production database. The code was developed and tested in
production against a new table structure, and it passed all its test.
Then when we went to roll it out, SQL errors all over the place. It
was quite a relief to be able to roll back the website with a simple
command. ( BTW, I'd really like a smart version control system for our
table structure, one that could do ALTER TABLEs appropriately when
deploying).

In our situation, we have a development server, and a production
server with a staging hostname and a production hostname. Once a
feature pasts testing on development, it goes out to staging. Staging
is on the production server and connects to the production database.
We do 'pre-live' testing there.

The great part about this is the seamless deployment from stating to
production. Our initial svn export gets linked to the staging
hostname. E.g.:
$ls
website-release-320
staging.website .com -website-release-320/
website.com -website-release-320/
$svn export http://svn-host/website/releases/release-345 ./website-
release-345
$rm -rf stating.website .com; ln -s ./website-release-345
staging.website .com
$ls
website-release-320
website-release-345
staging.website .com -website-release-345/
website.com -website-release-320/

Where staging.website .com was previously linked to ./website-
release-320 . Now the new code really is in the production
environment. There shouldn't be any problem in production that
wouldn't be apparent here. So, after we've done the testing at
staging, we do:

$rm -rf website.com; ln -s website-release-320 website.com
$ls
website-release-320
website-release-345
staging.website .com -website-release-345/
website.com -website-release-345/

The deployment of the new code to live takes milliseconds.

We can let the 320 revision hang around for a while; it's not being
served, and it's a "known good configuration" of the website
( provided it doesn't reference any outdated database structures :)
that we can rollback to if needed.

Finally, by using an adequately named directory to symlink to, I know
what version I have deployed as production. If /home/user/website/
production is a symlink to /home/user/website/trunk-R255, then there's
a pretty good chance that it's revision 255 that is currently in
production. If I were just copying files into the website directory, I
would have to have another way to know or guess what revision was
actually deployed.

You could use tags on your repository and then simply assume that it's
always the latest tagged version which goes production. When you reach
the next stable release, tag it and deploy it. Quite easy.

That's a good idea. We're suing subversion, which doesn't have tags,
but the manual says you get the same functionality by doing a release
branch.

Knowing myself and my computing habits, I wouldn't like to assume that
a directory is the latest tagged released; it may not be. ( My most
difficult bash-head-against-wall troubleshooting cases are always
faulty assumptions ). By creating a directory named trunk-R273 or
release-R503, then there's a greater likelihood that that directory
actually *contains* revision 503. Not that there's anything magical in
naming it as such, but hopefully when I'm improperly naming a
directory ( e.g. svn export http://svn-host/releases/R502 ./website-
release-R498 ), the cognitive dissonance becomes apparent, and I stop
and say "Hey, what am I doing? This is *wrong*!" For me at least,
there's a lesser likelihood of that happening when I'm doing export to
the generic directory name that we've been using for years.

[Charles Calvert]

On Mon, 13 Oct 2008 08:58:12 -0700 (PDT), la*****@gmail.c om wrote in
<12************ *************** *******@k7g2000 hsd.googlegroup s.com>:

>On Oct 13, 11:30*am, Charles Calvert <[e-mail] mwrote:

>On Mon, 13 Oct 2008 06:23:48 -0700 (PDT), lawp...@gmail.c om wrote in
<7edb880b-1c8e-4060-b59c-e2ecd2d0d...@v7 2g2000hsv.googl egroups.com>:

>On Oct 13, 12:06*am, Charles Calvert <[e-mail] mwrote:

>I think that I must have misunderstood, because it sounds like you
want to use Subversion as a packaging and distribution mechanism, and
that's not what it's for.

>Well, here's the problem, as I perceive it:

[snip]

>Users of the website need to be able to upload images, which in the
current incarnation of the website I specified to be served /home/user/
website/images. This directory currently holds all images served with
the site, not just user uploaded images.

>My 'packaging and distribution' practice currently consists of 'svn
exporthttp://svn-host/website/trunk', and then linking that to the
website directory, e.g. 'ln -s website-trunk-revision-189 /home/user/
website'.

I would just get the desired revision from svn and copy the source
files to the production directory. *You might need to clean out the
production directory first if you've removed any files.

Well, I can do all that in one fell swoop by creating a symbolic link.
I don't have to worry about whether or not I've deleted anything in
version control that I then need to remove in the production; it's
already done for me. If I had to delete files what were already
deleted in version control, then in my mind, I'm doing manual labor
that the computer should be doing anyway.

Also, by using symbolic links, I can deploy, test, and fallback to the
old directory by re-linking very easily if I need to.

Finally, by using an adequately named directory to symlink to, I know
what version I have deployed as production. If /home/user/website/
production is a symlink to /home/user/website/trunk-R255, then there's
a pretty good chance that it's revision 255 that is currently in
production.

This was the part that I was missing. It sounded like the working
copy was being used for production. You're not doing that. What you
are doing sounds reasonable.

[snip]

>You're making this too complex. Subversion is for controling
revisions of your source code and related files (e.g. design docs),
nothing else. Once you put the application into production and have
begun accumulating user data, just copy the source files from a
working directory to the production directory using cp -r or a shell
script.

Well, I didn't *intentionally* make it to complex; my assumptions
about website design and structure sort of led me to the place that
I'm in now. :)

I'm making a dymanimc website where users can input data. The source
code of the website is under version control, and the user-generated
data is stored in MySQL. So far, so good. If I needed to re-generate
the website for any reason ( backup, new webhost, errant rm -rf ), all
I needed was a recent export from subversion and the latest MySQL
dump.

Check. That's the same strategy I use.

>So now I think I'm beginning to understand the philosophy. In the
past, I could get away with keeping all the website images under /
images, or some similar structure.

Assuming that they are part of the application and not user content,
that makes perfect sense. After all, version control can work for
images too.

>But now that I'm accepting user data that I can't or shouldn't put
into MySQL, I need to find a third place to store things. I liked
the simplicity of having the whole website existing as svn export +
mysqldump, so I was hoping there was a way I could keep this new
kind of data within that system. But I can't. Now, the website must
be mysqldump + svn + some other thing.

Right. You could combine the mysqldump and the back-up of the
extradatabase user content into a single script and just tar
everything up together into a single back-up file. That should be a
fairly simple shell script, perhaps with the revision number as a
parameter that can be appended to the name of the tarball, much like
you're doing with the directories.
--
Charles Calvert | Web-site Design/Development
Celtic Wolf, Inc. | Software Design/Development
http://www.celticwolf.com/ | Data Conversion
[phone] | Project Management

[AqD]

On Oct 11, 3:25 am, lawp...@gmail.c om wrote:

I'm developing a php website that I have under subversion version
control. I'm working on an image upload functionality, and in the
middle of it I realized that any files that a user uploads will not be
under version control, and if I checkout or export the site from
version control, and deploy it, it won't bring any of the uploaded
files with it.

I'm looking at php subversion functions, but the manual says that
they're experimental, and my webhost hasn't deployed them.

What's a workaround for this? I'm thinking that I'll have to have a
parallel site for image hosting. For example, I'll have
images.website. com, hosted under a separate filesystem directory, and
then in the actual website that a user would peruse, any img src would
reference images.website. com . When I accept my uploads, I'll do my
filesystem copy to the images filesystem link.

Thoughts?

it could be done. Just call the "svn" commands in your code.

But it sucks in my experience to use large binary files (hundreds of
KBs) with subversion. The status (scanning / detecting) and exporting/
updating are painfully slow compared to direct copy.