On Thu, 3 Jun 2004, Artemio wrote:

Hello, Bryce!

Well, I have a pretty good experience with shell scripting and with PHP too, so if these two will be enough - you can count on me.

There are several questions I want to ask:

1. As I understood, the scripts should work on the SourceForge server or you

meant that someone should provide his own server for that?

We don't need someone to provide a server. So SourceForge would be an appropriate solution, but I'm open to better ideas if someone is familiar with other mirror systems, as long as the result is something that does not depend on any single person or company (i.e. we want to avoid single point of failures).

2. <quote>Allows users a place on the web to go to to get the latest version

</quote> - where would this place be? At inkscape.org?

Yes, or if the files were uploaded to the SF file tracker, then it could be just that file listing. Basically I just mean, "We need a URL to point to from the download page" so people can easily navigate to it in their browser. So it doesn't need to be anything very fancy, it could be just a webpage with links to the snapshots.

3. <quote> Doesn't place the administrative responsibility on a single

specific person </quote> - can't get what you mean here exactly.

Okay, basically there's two points. First, we don't want to have to burden just one person with the work of having to maintain and administrate it. For example, we'd like a situation like we have with the website, where anyone in the project can log in and update things as needed. This way nobody feels burdened by having to be responsible for it. Second, and more importantly, we don't want to have a single point of failure. If the person(s) maintaining it goes on vacation or gets tied up with other things, we need other project members to be able to step in and quickly and easily fill in the hole.

I'll be glad to help if I can. I can help with scripting but I don't have my own machine to do all that compilation stuff. I am pretty familiar with sourceforge's server project file hierarchy so I'd probably be able to handle this task - at least to try ;-)

Great, thanks! Sounds like you have the know-how to do it. Let me know your SF username and I'll add you to the project.

Do you know CVS? You can check out the inkscape_project module and create a subdirectory under that for this work.

You should probably also talk with Bob and review his setup and scripts; possibly you can reuse and/or build on his work, at least for the Win32 port. Hopefully he has time to help with advice, etc. too.

Bryce

On Fri, Jun 04, 2004 at 12:26:28PM +0300, Artemio wrote:

1. Get the latest inkscape from cvs.
2. Compile it for linux, win32 and mac.
3. If the compilation is successful, prepare the source snapshot, and builds

snapshots in .tar.bz2 and .tar.gz.

The source snaps are already pushed nightly so only the buids need to be packaged. In the "inkscape_project" CVS module, under the "tarball" directory, you can see "update.sh":

http://cvs.sourceforge.net/viewcvs.py/*checkout*/inkscape/inkscape_project/t...

The binary compiles would most likely be:

- win32: just continue to do what Bob is doing and make a ZIP of the entire build. - linux: build RPMs for a variety of distros (suse, fedora, mandrake), build debs for debian unstable.

4. Move these archives to $somewhere.

$somewhere would be on the sourceforge server, similar to the nightly source trees.

5. Generate a web page with links to these snapshots.

Like the source snaps, this can just be a directory people look into. The nightly sources go into ${WEBDIRECTORY}/cvs-snap so maybe the builds would go into ${WEBDIRECTORY}/build-snap with "debian", "RPM" dirs?

1. Where would the script run? At the sourceforge server?

I don't think this part can be run on the sf servers. There aren't compilers (or libraries) on the shell server, and the compile farm machines usually don't have all the libraries needed for more complex packages. (And I'm not sure that the compile farms can be easily "automated".)

I would recommend having individuals build the snapshot locally and then push the results to sourceforge. Setting up an SSH key to the sf server is well documented:

http://sourceforge.net/docman/display_doc.php?docid=761&group_id=1

2. I don't know (yet) how to prepare a binary build in an archive. I mean that

make install puts everything in --prefix, but should the script "manually" get the data? Or there is a list of files to install and I can read it and use to prepare an archive?

There is a predefined way to build both a .rpm and a .deb file, so those packages can just be built "normally". I'm not 100% clear on the win32 process, though. (More details below.)

3. Is the target snaphots dir and the target web page with links on the same

server that the script runs?

That would be the best, but I don't think we can do it with sourceforge (which is where the current source snaps are held).

Sorry, if you feel I am lame here (question #2) you may discard my help. But if I get any hints/suggestions, I think I'll make it ;-)

I should probably give more details to my answer to #2. Have you done any RPM or deb packaging before? With RPM the best way is to probably build the source .tar.gz first (via "make dist"), then:

rpmbuild -ta inkscape-....tar.gz

to build debian, it needs to be first unpacked:

fakeroot dpkg-buildpackage -us -uc

All *nix builds should do a "make distcheck" before attempting to compile. If "diskcheck" fails, the binaries shouldn't be distributed, IMHO.

so my pseudo code would be:

- cvs checkout - ./autogen.sh - ./configure - make distcheck - make dist - rpmbuild -ta ....tar.gz - fakeroot dpkg-buildpackage -us -uc - scp to sf

(although I don't think 1 person can do the builds for all the RPM flavors...)

On Fri, 4 Jun 2004, Artemio wrote:

Hi all!

Okay, now it all seems clearer for me.

So, the task is:

1. Get the latest inkscape from cvs.
2. Compile it for linux, win32 and mac.

Also, compile both debug and optimized versions. I sense that users want both - the debug version to get extra info when reporting bugs they find, and the optimized version for actual use.

3. If the compilation is successful, prepare the source snapshot, and builds

snapshots in .tar.bz2 and .tar.gz.

Actually, you'll want to build the rpm and exe packages. If you're unfamiliar with packaging, no problem - a bunch of us know this very well and can teach and/or help with this part. It's pretty easy and well documented.

4. Move these archives to $somewhere.
5. Generate a web page with links to these snapshots.

Here are my questions:

1. Where would the script run? At the sourceforge server?

Yes.

2. I don't know (yet) how to prepare a binary build in an archive. I mean that

make install puts everything in --prefix, but should the script "manually" get the data? Or there is a list of files to install and I can read it and use to prepare an archive?

Don't worry, we've had a lot of practice at doing this and have documented the process, so you'll need to learn it but it is pretty straightforward, and there's plenty of people who can help. You can look on the Inkscape wiki on the CreatingDists page, which should have ALL the info you need.

To create the dist version of the tarball, after you've run configure and make, and set the version number, you can run `make dist`, and that will generate a dist tarball.

Once you have a dist tarball, you run the rpm build command, and that'll go off and run some compiles and tests and generate an rpm for you. This is going to be the hardest part for you to learn, but fortunately there's a lot of us who are very familiar with this and can help directly. Ted and myself do this for the releases so will be able to answer questions on it.

For the windows builds, I am less familiar with the process, but I'm certain Bob will be equally ready and available to provide training and assistance. This process is also documented so I expect it will be straightforward to set up.

For Mac I'm even less familiar, but am aware of some documentation on it. I'm not sure who the best resource person is to help with this, so that might take some digging. Since we don't currently provide nightlies for MacOSX, I'd suspect this would be a lower priority than the other two platforms. However, I also suspect we'd gain as much value if we have nightlies as we've gained from the other platforms.

3. Is the target snaphots dir and the target web page with links on the same

server that the script runs?

I don't know, but suspect that potentially you'll be running scripts on two different machines. Also, note that the Inkscape web project directory is limited to 100mb, and we're already using up half that space, so we wouldn't be able to store very many packages without running out of space - this is why I strongly suggest that the nightlies be posted to some file server / mirror system, such as the SF File tool, or other such place.

By the way, it would probably be very wise to spend an hour or two looking through Freshmeat & google. on the off chance that someone else has done this before and have some scripts/ideas/processes/docs you could beg, borrow, or steal to save you some time.

Sorry, if you feel I am lame here (question #2) you may discard my help. But if I get any hints/suggestions, I think I'll make it ;-)

These are exactly the right questions to ask. You'll have a lot to learn for making packages, and it can be tricky, but fortunately like I said it's pretty well documented and a lot of people know how to do it for one platform or another and can give advice (or at least moral support!) when you get stuck. None of us have automated building on all these systems, so it will be an impressive achievement once you've got it working!

Bryce

On Fri, 4 Jun 2004, Artemio wrote:

Hi all! Hi, Bryce! ;-)

Okay, I got it. I am reading the CreatingDists page at Wiki ;-)

Here are my next questions:

1. You (Bryce) say that the script will run on SF's server. But someone said

that it has no build tools installed... So?

Yeah, sorry for the conflicting answers. Take this as an indication that you've reached the point on that topic where we don't really know, and you'll have to do some experimentation.

It is correct that the SF web and shell servers don't have gcc, etc. and don't like running builds from them, as Kees said. The build farms do have these tools, but I don't know if they permit automated builds - Kees says they don't and he's probably right, but I'd suggest looking into it and seeing for yourself. If it is possible to automate builds on them, then that would be a good solution.

This uncertainty is why I was a bit ambiguous upfront about what server(s) this would need to run on. You'll need to investigate and experiment and see what can be done. If it can't be run on the SourceForge machines, then there are probably other solutions that can be found.

4. You were telling about the lack of web space. Well, we can actually try to

do the following: I have around 60-80 free megs at my fantomized.info site and my hosting company provides free traffic. If you are able to create a subdomain like download.inkscape.org which would point to the IP I'll tell you, I'd add this domain (download.inkscape.org) to my addon domains and this site would simply work from my server account. How's this idea?

It's a good idea, although remember that it's a very important requirement that access and maintenance of this not be limited to a single person, as mentioned before, so that could be tricky if it's a personal account. But yes, looking at solutions beyond SourceForge is a good idea if SF can't provide what we need; we just need to make sure that anyone on the project can login and fix stuff as needed.

Bryce

Daniel Díaz wrote:

I've been playing with the autobuilding stuff for Windows. I already downloaded the cross-compiling stuff and it works OK on my system! (It's a Mandrake 9.2-10.0).

Excellent! I was hoping that there would not be any outstanding dependencies.

Now, what should be in the .ZIP file? I have this .EXE, but what else? I've looked in the Wiki and Bob's page, but couldn't find anything about that.

In the /inkscape top directory, type make -f Makefile.mingw dist

And it will create a working Inkscape/w32 directory: /inkscape/inkscape

This is what should be zipped or packaged.

I remember there was an installer as well, but since I have no Windows boxes around, I can't really tell how's that.

Look at NDIS on Sourceforge.net. We used it for the last two releases, and there is an /inkscape/inkscape.nsi project file in CVS. This is a lot slower than a zip file.... you might not want to use it every night, especially as people can extract files from .zip individually if they want.

Thanks for looking at this.

Bob

Hello all.

On Mon, 2004-06-07 at 08:17, Bob Jamison wrote:

Daniel Díaz wrote:

...

I've been playing with the autobuilding stuff for Windows. I already downloaded the cross-compiling stuff and it works OK on my system! (It's a Mandrake 9.2-10.0).

Excellent! I was hoping that there would not be any outstanding dependencies.

I only got one problem with a #include_next, but it was easy to fix.

...

Now, what should be in the .ZIP file? I have this .EXE, but what else? I've looked in the Wiki and Bob's page, but couldn't find anything about that.

In the /inkscape top directory, type make -f Makefile.mingw dist And it will create a working Inkscape/w32 directory: /inkscape/inkscape This is what should be zipped or packaged.

Got it! Then I checked your scripts and it was all there! Mines are very closely following yours.

I've been trying to automate this stuff, and seems like it's working OK! There are many things to be done, but this ought to make it for now: http://cortijodelrio.net/~inkscape/

It's building three times a day: at @000, at @333, and @666 (Internet Time).

The system is made in three stages with the help of some cronjobs: "updatecvs" ran one hour before and 10 minutes before building time; "compile" which does what it says; and then "makepkgs", which takes care of the .tar.gz and .zip files.

I can add another stage called "sync", which would take care of scp'ing the archives somewhere else if needed. If anyone is feeling like a mirror these days, just drop me a note.

If anyone wants to check that autobuilding process, just send me a private message (or your SSH key) in order to log in to the server.

Greetings!

Daniel Díaz yosoy@...31...

Yup, I mentioned that as a possible solution. Well worth looking into further.

Bryce

On Thu, 3 Jun 2004, Jon Phillips wrote:

What about sf.net's render/compile farms?

Jon

On Thu, 2004-06-03 at 11:55, Bob Jamison wrote:

...

Bryce Harrington wrote:

...

Hi all,

As Bob mentioned, the visit from Slashdot cratered our nightly Win32 snapshot server, and we need some help coming up with a better solution. No Inkscape experience needed, just need to be familiar with SourceForge, scripting, and compiling apps.

The pages are not gone. Just replace /inkscape/ in the URL with /.inkscape/ for now. (Please don't tell anyone!)

...

The snapshots have proven useful in letting users test out and report on recently implemented features without having to compile it themselves. So this is a pretty highly valued service for the community.

What we need going forward is a script or set of scripts that:

• Checks out from CVS once a day and compiles for Win/Lin/Mac both debug and optimized. (e.g. via SF compile farm?)
• If compilation is successful, post the packages to mirrors (e.g. SourceForge).
• Allows users a place on the web to go to to get the latest version
• Creates and provides autopackage versions (the AP folks are doing this for us currently, so this may not be necessary but is worth considering if it's not hard to do.)
• Doesn't place the administrative responsibility on a single specific person
• Code is checked into a subdir of inkscape_project/
• Has a README and/or wiki page explaining install & usage

If you'd like to work on this or a piece of it, let me know and we can work out the details.

Bryce

Here is the simple script that I have been using.

Note that I am using CVS/SSH to download a current copy. The anoncvs servers are many hours older than current. The site says 5 hours, but it is much more than that, I'm sure. If you use :pserver:, then this will be easy. If you want SSH, what you will need first is to generate a DSA SSL key, upload it to SF, and configure your box so that you can SSH to the cvs server without a password. SF.net has instructions about how to do this.

Notice that the script tries to checkout 10 times before quitting. CVS often fails the first couple of times.

The XMingW32 cross-compiler and gtk2.zip are on my site. They should be all you need to auto-build on linux.

######## SNIP SNIP ############

######## nightly.cron ############ # Crontab for automated builds MAILTO=YOURADDRESS # # Every night at ten minutes after 3am 10 3 * * * (cd /home/YOURNAME/xmingw32/nightly ; /bin/bash autobuild.sh )

##### autobuild.sh #########

#!/bin/bash

PATH=/usr/local/xmingw32/bin:$PATH

# In case we are called by another user HOME=/home/YOURNAME

rm -rf inkscape

# CHECKOUT

a=0

while [ $a -le 10 ]; do

echo "##### Checkout attempt $a" a=$(($a+1))

# #### PICK ONE ### # ANONCVS # cvs -d :pserver:anonymous@...54...:/cvsroot/inkscape export -r HEAD inkscape

# CVS/SSH env CVS_RSH=/usr/bin/ssh /usr/bin/cvs -Q -d :ext:YOUR_SF_ID@...54...:/cvsroot/inkscape export -r HEAD inkscape

if [ -d inkscape ]; then break fi

done

if [ -d inkscape ]; then echo "##### Checkout success" else echo "##### Checkout failure ... abort" exit 1 fi

# BUILD

cd inkscape

/usr/bin/make -f Makefile.mingw

/usr/bin/make -f Makefile.mingw dist

# PACKAGE

datestr=`date +%y%m%d%H%M`

zipfile=Inkscape${datestr}.zip

/usr/bin/zip -r -q -9 ${zipfile} inkscape

builds=${HOME}/public_html/inkscape/builds nightly=InkscapeNightly.zip

/bin/mv ${zipfile} ${builds}

( cd ${builds} ; /bin/rm -f ${nightly} ; /bin/ln -s ${zipfile} ${nightly} )

---

This SF.Net email is sponsored by the new InstallShield X.

...

From Windows to Linux, servers to mobile, InstallShield X is the one

installation-authoring solution that does it all. Learn more and evaluate today! http://www.installshield.com/Dev2Dev/0504 _______________________________________________ Inkscape-devel mailing list Inkscape-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/inkscape-devel

John Cliff wrote:

Hi guys, Hadnt read the mail, but saw bob asking for someone to put a build on SF in jabber. SF say to use the webspace rather than file release for nightlys, so I've compiled a fresh checkout, and added the zip to the inkscape_web CVS module. can someone run the publish_web.sh script for me? I've no idea how to go about doing it, and its 2am here so I really need to get to bed (start work in 6hrs) I know this isnt as good a solution as the full automatic nightlys, but its a start :) file is inkscape04-06-04.zip in root of inkscape_web the wiki and dl pages will need changing too.

Finally got through. Thanks sim.

njh

On Fri, 4 Jun 2004, Mike Hearn wrote:

On Thu, 03 Jun 2004 11:19:28 -0700, Bryce Harrington wrote:

...

• Creates and provides autopackage versions (the AP folks are doing this for us currently, so this may not be necessary but is worth considering if it's not hard to do.)

Until autopackage 1.0 is out I'm happy to maintain the nightly .packages: they are compiled on navi at midnight Colorado time, and I get notifications of each (un)successful build. Only stripped builds are available at this time.

Okay, perfect. Especially since autopackage is still pre-1.0, that seems like a good arrangement. Maybe later on if/when it makes sense for you to turn over maintenance of it to us, we can integrate it into an all-in-one solution.

Currently bandwidth usage is not a problem as they aren't linked to from anywhere. If it becomes troublesome in future I can upload them to a university server which should have more bandwidth (or to SourceForge for mirroring, I don't mind).

Okay, cool, keep an eye on it and let us know how it goes.

Thanks, Bryce