My new rating system - the Alpha version...

[BobTrips]

I've been trying to figure out how to award points/Smileys. I'm trying out a system where I will award one point to a picture that I feel shows "artistic merit", that the photographer saw a good picture in terms of beauty, novelty, a refreshing angle, ....

I will award a second point for "technical merit", the photographer's work in capturing the image and preparing/presenting it on the site.

At the bottom of the critique one might see (V=1, T=1). That would be a "Big Smiley".

Already I'm feeling for a bit more latitude, maybe up to two points per.

Any feedback? Make sense to anyone?

[AdrianW]

I might be tempted to split both technical and artistic into two.

Artistic subchoices:

Good composition?

Uniqueness/Novelty?

Technical subchoices:

Photographic technical - nice use of depth of field/shutter speed etc

"Post" technical - sharpening, horizons correct, contrast/brightness well chosen etc

I probably prefer the option of four different levels for each - that way people can't always hit the middle ;-)

[BobTrips]

Well, I'd actually like something more like a 0 to 5 rating on each of the two major categories. I'm just trying to find a more objective method for me to assign Smileys.

At first I just gave everyone one point - A "Simple Smiley".

After a while I saw a picture that I thought was pretty bad getting "Big Smileys" for reasons along the lines of "Ohhhh, I love being on the beach, this reminds me of a time when I was ...." For the first time I dispensed the "Dreaded Yellow Scowl".

The I started recognizing shots that I thought were way above average. I went back and upgraded several Simple Smileys to Big Smileys.

This little exercise is aimed at further sharping my system. I figure that I can attempt to communicate the subtleties in the text.

[ronners]

Bob, your last sentence is key. Contrary to other suggestions, I would prefer to simplify the current system, or leave it as it is. If you like it, you give it a point, if you don't like it, you don't give it a point. Use the 'comment' field to provide an explanation.

Introducing a complex rating system will, in my view, reduce the number of critiques rather than increase them by introducing an unnecessary barrier of complexity. Rating systems are normally used to provide a consistent frame of reference for critiques. Ultimately, does this then mean that my 'simple smiley' is equivalent to your simple smiley? Probably not. My criterion for a critique may well be different than yours, and thus the usefulness of a statistically-based appraisal system is inversely proportional to its complexity.

As I alluded to in an earlier post, one of the nice things about TrekEarth is that it helps us all improve our skill, by helping us figure out what we do and don't like about photographs. It's not, however, about statistics or convoluted 'scoring' mechanisms. Does it matter that one photo I like has fewer points than another I don't? Not always.

An admirer of a photograph shouldn't feel obliged to justify their admiration. So, as I say, let's keep it simple. Any point system should only serve to possibly direct members to photos worth consideration, nothing more.

Ron.

[AdrianW]

Ronners: Actually I disagree. Finding out what areas the picture fell down would help one improve things. For example was it a perfect shot, but cliched out of existence - or was it really exciting but ruined by lack of technical skill?

[ronners]

But which is a better approach Adrian - to learn from others through their work, and 'inherit' their skill, or have a subjective notion of technical excellence imposed on your submissions? I believe that actions speak louder than words - or indeed points.

[BobTrips]

I think Ron made some good points in a couple of posts back. My preference would be to have no Smileys or points. I don't like awarding points.

I'd rather just comment on shots. But one has to click on one of three ratings the way the system is designed. I devised my 'Alpha' system to help me figure out a way that 'I' could be comfortable within the existing system.

BTW, I don't have a huge problem with the existing system. I just couldn't figure out what was a 1 and what was a 2. I've been trying out a system of one for image, one for technique and wondering how others might react.

I expect to get insightful comments on my shots from Adrian and others on technique. I expect to get insightful comments on content from people who don't know "jack" about technique. Maybe my system wouldn't work for people who don't have a good grasp on one of the two aspects?

And, no, I don't want to get into a well defined measurement system. I spent a lot of my professional and research career trying to figure out how to measure difficult stuff. I don't really want to go there again.

(I do like this discussion, however ;o)

[BobTrips]

"My criterion for a critique may well be different than yours, and thus the usefulness of a statistically-based appraisal system is inversely proportional to its complexity."

Actually a properly designed measurement system deals with these sorts of problems fairly well. I could write a chapter or two on how this is done.

But, I don't want to.

[Skorj]

The photographer in me appreciates the -critical- comments about the technical content of my work and how to improve it (particularly never having been trained as a photographer).

The egotistical artist in me apppreciates the points.

[sita4ram]

Well; I totally am for the method bobtrips suggested. I think everyone who browsed through the gallery notices some great pictures with the wrong light, blurred etc. and that have got no critics whatsoever, even though they are of good vision and make a point.
here is what I think hapening... cause thats the way it goes with me as well...
I see the thumbnail, click on it to get a better view of the photo, and then the inside looks like an enlarged thumbnail - no details, bad crop, no light, and even though its an interesting one, I dont wanna start writing all that is wrong with that cause it can take ages, and it can also sound bad - I mean too many 'wrongs' in one photo. and I dont wanna give it one point because as a photo its not good.
so I think this is what happens with most of those photos and most of the critics. I think a system of technique and vision is very reational and benefits more people.
but hey, its just me. maybe Im wrong.

[adam]

"but hey, its just me. maybe Im wrong."

There is no right or wrong. You make some good points. If enough members want such a system I can always change it.

[BobTrips]

After using my 'Alpha' system for a while I would give it mixed reviews.

It's excellent (IMHO) in terms of helping me focus my attention separately on content and technique. I'd be in favor of some sort of suggestion to take a look at both aspects of a picture that cropped up whenever someone chose to write a critique.

I doesn't always work. It especially doesn't work when I sometimes take a look at the first few posting from a new member.

Some people post pictures that aren't all that good, both in terms of vision and technique. I'd like to give them some ideas of how they might improve their work. I don't want to punish them with the Nasty Yellow. I make some (hopefully) helpful comments and give them a Simple Smile.

Again, (and I'll keep pounding on this) I don't like the point system. At least I don't like the public posting of points. I would prefer that the element of competition be minimized.

I don't see this site as a contest. At least I don't want to see it turn into one.

There's already (what looks to me) like a race to see who can post the most pictures. Some people dump '3 per' on the site and don't go back to rework their posts after receiving feedback.

[philip_coggan]

There are several photo sites around, and they all seem to have a points system. Photonet suffered near-terminal meltdown because of it, and photosig has had ongoing ructions. Personally, I'd like to see no points, faces etc at all - just photos and comments.

If there are souls who just just can't help being competitive, give a point for each photo and a point for each comment, irrespective of merit - that would at least encourage particiaption (tho not quality :-).

[adam]

"I don't want to punish them with the Nasty Yellow."

I changed the nasty red into yellow to soften the negativity and now yellow is becoming nasty? I may have to change nasty yellow to a very pale green as suggested by Viviane a while back.

[adam]

Photonet had a points system? I never knew that. I thought they used averages.

The reason I have points is for the computer to know which ones to place on the main TE page. And it makes things more interesting/fun for many people. However I am aware of the problems with a point system. I could eventually substitute using Favorites as a way of displaying the top photos, but not now. One thing to note is that you 'get' absolutely nothing for having points on this site, ie one's daily upload limit is not increased, etc. I believe this minimizes some of the negative aspects of the point system.

[BobTrips]

It doesn't matter what color you make the Nasty Yellow. If you make it pale green it will become the Nasty Pale Green. It's a 'In Your Face Zero. In many/most circumstances it will be/is a punisher.

I can understand your desire to have some sort of a rating system to aid you in choosing highly regarded images for the front page. The 'Favorites' method (without the number of people listing the image as a Favorite) would work.

(I also understand your desire to wait for a while to make this change. Let's see the Beta version ASAP ;o)

If you would like to stick with some sort of scoring system let me suggest:

a) Awarding 1 or 2 points. Just display a Simple Smiley or Big Smiley, don't display anything when the critiquer doesn't award points.

b) Don't publicly report point totals. When you total points received you start to rank photographers and create competition.

(And the current system is very problem ridden. I'm saying that as someone with an extensive background in designing measurement systems).

[AdrianW]

OK, I'm afraid I'm going to disagree with the idea of not giving points. Points enable me to see what other people think of my images, which in turn gives me an idea of what works and what doesn't. I really enjoy it when I get two points, because it means that someone else agrees with my "vision" and the way I presented it!

[sita4ram]

I think it doesnt matter if you get points that add up or just small/big smile.
and Adam - if you wanna keep that nasty yellow, try to make it smile with sympathy :)
I mean there are all sort of smily faces. you can get ideas from msn. maybe its the face of that yellow that Bob ant others try to avoid showing :)

now about the points - well you can keep them but not display them on the gallery page. (just a suggestion) maybe each user can see his critiques, but it doesnt show next to his name?
The system as it works right now is fine as well... and of course before you make changes based on 2-3 ppl, make sure the rest are into it too.

[mdchachi]

I think the points are useful; it provides a quick measure of which of my photos which have had the most impact. The total points for each member is somewhat useful in terms of showing me which photographers are newer to the site and may need some encouragement. Obviously it doesn't tell you which photographers are the "best" because those who have been here longer or have uploaded more photos will have higher totals. One way to measure something like that is to divide a member's total by the number of photos -- ie average points per photo. I'd be kind of interested in seeing that just to see where I rank. It might make the site more competitive as some people fear. On the other hand, it might help encourage people to remove mediocre photos or not to upload them in the first place (thereby raising overall quality).

As for the "nasty" yellow face, I think it's good to have. We need some (gentle) way to discourage poor photos so that people don't start using this site to display their generic travel snapshots.

[adam]

For the time being I'm going to focus on allowing comments in addition to critiques.

[sita4ram]

Well I think so far its the best solution. seems like its the only idea everyone agrees on, and it allows one to say something without having to put a 'score' stamp on the picture.
Good idea, in my opinion.

[BobTrips]

I think this moves closer to an acceptable (to me) scoring system. It would produce a more meaningful number.

As it stands a photo which receives 15 '1's displays as 'better' than one that receives 3 '2's.

(Often when a really excellent photo shows up and receives a few excellent ratings I don't bother to 'pile on' by adding another 2, there's nothing else to say.)

But, an averaging system would need another tweak. We get people becoming members just to assign points to a single picture and then disappearing. I can imagine some 'point hungry' person lining up all their friends at a party, requiring them to sit down, register, and assign Big Smilies to their single mediocre image. Now we've got a 'Perfect Two' against one of Nina's wonderful NZ images which gets only a measly 1.8.
-----
All of that aside, one solution to the competition issue is to display the cumulative points no where except on the individual photo page.

(Perhaps in the "My Account" section there could be a ranking of thumbnails by average score for the individual photographer to see.)