Pet Spotlight: Community Review Pilot
Hey everyone! We're trying something new with the Pet Spotlight, and we need your help to test it.
The Problem
The spotlight queue has gotten really long, with some of you have been waiting years for your pet to be reviewed. That's not fair to anyone, and it's not sustainable. I want to fix that!
The Idea
Instead of staff being the only ones who review pets, we're opening it up to the community. You can sign up as a Spotlight Judge and help review nominated pets. Esmerelda — yes, that Esmerelda — is running the show.
How It Works
For Judges
- Visit the Pet Spotlight and click "Review Pets"
- Click "Become a Spotlight Judge" to opt in
- You'll be shown pets to review — up to 5 per day
- For each pet, you check off criteria:
- Visual Design
- Story / Character
- Treasure Chest (does it fit the vibe?)
- Minion (does it fit the theme?)
- Name
- Completeness
- Give an overall star rating (1-5)
- Leave optional written feedback for the owner
- Earn 50,000 sP per review
- Work toward four new Spotlight Judge achievements
For Pet Owners
- Your pet gets reviewed by multiple community members
- You'll receive detailed feedback on what's working and what could be improved — delivered personally by Esmerelda (she has opinions)
- Pets that pass community review go straight into the winner queue
The Rules
- You can't review your own pets
- You can't review your friends' pets
- Your reviews build a reputation score over time — reviewers whose scores align with the consensus gain more influence
- Pets need agreement from multiple reviewers to pass or fail (no one person decides)
This Is a Pilot
I'm starting with just 5 pets from the existing queue. This is a test run to make sure the system works and feels fair before we open it up to everyone.
We will honor the results. Whatever the community decides for these 5 pets, that's what happens.
We Want Your Feedback
After you've tried it — whether as a judge or a pet owner — please reply to this thread and tell me:
- Was the review process clear and easy to follow?
- Did the criteria make sense?
- Did the feedback feel useful (for owners)?
- Was the sP reward fair?
- What would you change?
Your feedback will directly shape the final version of this feature.
What's Next
If the pilot goes well, I'm planning some improvements and changes to the system:
- Seasonal themed spotlight competitions tied to holidays (Morostide, Luminaire, etc.)
- Top 3 seasonal winners with special trophies
- More rewards for active judges (reactions, avatar borders)
- Opening the full queue to community review
- Potentially moving back to more winners being drawn more frequently (we've got over 300 pets waiting to be judged!)
Head over to the Pet Spotlight to check it out!
The Problem
The spotlight queue has gotten really long, with some of you have been waiting years for your pet to be reviewed. That's not fair to anyone, and it's not sustainable. I want to fix that!The Idea
Instead of staff being the only ones who review pets, we're opening it up to the community. You can sign up as a Spotlight Judge and help review nominated pets. Esmerelda — yes, that Esmerelda — is running the show. How It Works
For Judges
- Visit the Pet Spotlight and click "Review Pets"- Click "Become a Spotlight Judge" to opt in
- You'll be shown pets to review — up to 5 per day
- For each pet, you check off criteria:
- Visual Design
- Story / Character
- Treasure Chest (does it fit the vibe?)
- Minion (does it fit the theme?)
- Name
- Completeness
- Give an overall star rating (1-5)
- Leave optional written feedback for the owner
- Earn 50,000 sP per review
- Work toward four new Spotlight Judge achievements
For Pet Owners
- Your pet gets reviewed by multiple community members- You'll receive detailed feedback on what's working and what could be improved — delivered personally by Esmerelda (she has opinions)
- Pets that pass community review go straight into the winner queue
The Rules
- You can't review your own pets- You can't review your friends' pets
- Your reviews build a reputation score over time — reviewers whose scores align with the consensus gain more influence
- Pets need agreement from multiple reviewers to pass or fail (no one person decides)
This Is a Pilot
I'm starting with just 5 pets from the existing queue. This is a test run to make sure the system works and feels fair before we open it up to everyone.We will honor the results. Whatever the community decides for these 5 pets, that's what happens.
We Want Your Feedback
After you've tried it — whether as a judge or a pet owner — please reply to this thread and tell me:- Was the review process clear and easy to follow?
- Did the criteria make sense?
- Did the feedback feel useful (for owners)?
- Was the sP reward fair?
- What would you change?
Your feedback will directly shape the final version of this feature.
What's Next
If the pilot goes well, I'm planning some improvements and changes to the system:- Seasonal themed spotlight competitions tied to holidays (Morostide, Luminaire, etc.)
- Top 3 seasonal winners with special trophies
- More rewards for active judges (reactions, avatar borders)
- Opening the full queue to community review
- Potentially moving back to more winners being drawn more frequently (we've got over 300 pets waiting to be judged!)
Head over to the Pet Spotlight to check it out!
Posted by Keith
Load this on 🆕 Kumos site
I'm very excited for this and giving it a shot; will leave some feedback on the process after poking around a bit with it!
Here are my answers to your questions, as far as I can give them without being able to submit the review:
Was the review process clear and easy to follow? Yes, it was simple and straightfoward.
Did the criteria make sense? Yes, although as someone who's entered the Spotlight a lot before, I was already familiar with them.
Did the feedback feel useful (for owners)? I do feel like the written portion shouldn't be optional, since just the checkboxes wouldn't give me much information as an owner. (For instance, if the minion box wasn't checked, I'd like to know what about the minion didn't fit the pet, maybe even suggestions for more appropriate minions if the reviewer had any.) Maybe there could be (briefer) written feedback boxes for each of the criteria with prompts that are required if the box isn't checked, such as "How could the minion be more appropriate?"
Was the sP reward fair? I didn't get the reward since it wouldn't let me submit, but 50,000 sP sounds fair to me. Although I'd review pets for free so maybe don't go by my opinion ^^;
What would you change? Other than making it so that it doesn't show me pets that can't be reviewed (assuming that's what happened with the error I got), the only thing I'd change is the written feedback as described above.
Was the review process clear and easy to follow? Yes, it was simple and straightfoward.
Did the criteria make sense? Yes, although as someone who's entered the Spotlight a lot before, I was already familiar with them.
Did the feedback feel useful (for owners)? I do feel like the written portion shouldn't be optional, since just the checkboxes wouldn't give me much information as an owner. (For instance, if the minion box wasn't checked, I'd like to know what about the minion didn't fit the pet, maybe even suggestions for more appropriate minions if the reviewer had any.) Maybe there could be (briefer) written feedback boxes for each of the criteria with prompts that are required if the box isn't checked, such as "How could the minion be more appropriate?"
Was the sP reward fair? I didn't get the reward since it wouldn't let me submit, but 50,000 sP sounds fair to me. Although I'd review pets for free so maybe don't go by my opinion ^^;
What would you change? Other than making it so that it doesn't show me pets that can't be reviewed (assuming that's what happened with the error I got), the only thing I'd change is the written feedback as described above.
@Keith I signed up as a reviewer and wrote a review for the pet it showed me, but when I hit "Submit Review," I got a message that said "This pet is not currently in community review," even though it's the pet that was assignmend to me. The pet was Battle Warrior. I'm saving my review in a notepad file in case I can submit it later!
Was the review process clear and easy to follow?
Yes; I especially appreciated that star ratings had suggestions for what a rating could mean. Having 3 be the "passing grade" was a nice way of indicating a profile's readiness, as well as indicating which profiles go above and beyond, without necessarily placing a value judgement on another person's work. I'm a softie and tend to be very forgiving with assessment (as an actual writing instructor lol), so a reminder that a 3 isn't just an "eh" was helpful for me.
Did the criteria make sense?
Yes, I thought so. Criteria felt clear without being overwhelming in terms of what to consider when reviewing profiles.
Did the feedback feel useful (for owners)?
I can't comment on this, since I haven't received feedback, but I hope so!
Was the sP reward fair?
OK admittedly this is tricky for me. I LOVE doing stuff like this. Adore it. I love giving feedback on people's creative works and helping their intentions and visions shine. I would do this without an sP reward. But my enjoyment of the process itself isn't necessarily accurate to what other users feel, so I'm trying to put myself in the shoes of the other. I think it could stand to be bumped up a bit--maybe to 100k?--because reviewing a profile is a lot of work, especially when the review needs to be concise. You have to be able to identify exactly what to focus on to provide feedback, and in at least one profile I reviewed, I didn't mention small things that probably should be changed because I didn't have the space or personal time to revise my review to mention those small elements without sacrificing a more meaningful review about more important elements of the profile. My point is, that require thought and experience in this type of work, and 50k in virtual currency is, well, not much at all!
What would you change?
A few things. Let me try and be concise here with my suggestions, and these are presented in no particular order--just what came to mind while I'm typing.
1. Provide two different feedback sections--one larger that focuses on the "big picture" (i.e., the most important and meaningful parts of a profile), and a smaller one for tiny things that should be adjusted (such as grammatical errors, aesthetic tweaks, things like that). The smaller section should probably top out at 200 characters max. I think the larger section can be kept at 1000 characters.
2. Allow users a means of rating feedback to reviewers. Something as simple as a Likert scale would be helpful, or a section for brief comments about what was helpful. These ratings could then influence a reviewer's reputation, though I'm not sure by how much.
3. Increase sP payout based on reputation.
4. Provide some more in-depth guidelines for reviewing creative works. I would be happy to help provide my input on this if such a thing was added--again, speaking as someone who professionally teaches and assesses writing/creative work broadly, in both publishing and pedagogical contexts. If such guidelines were implemented, they should probably be presented on a page before signing up to be a reviewer, and then would be linked on the review page for later reference as needed.
Sincerely, this is a really fun way for me to engage with the site and community. Thank you for putting it together! It'll probably distract me from grading my students' work, but that's completely OK. :)